Jun 11 22:49:52.425049 ip-10-0-138-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 11 22:49:52.425066 ip-10-0-138-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 11 22:49:52.425076 ip-10-0-138-236 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 11 22:49:52.425495 ip-10-0-138-236 systemd[1]: Failed to start Kubernetes Kubelet. Jun 11 22:50:02.498613 ip-10-0-138-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 11 22:50:02.498629 ip-10-0-138-236 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f2044582d61342158b44262addebace4 -- Jun 11 22:52:27.958142 ip-10-0-138-236 systemd[1]: Starting Kubernetes Kubelet... Jun 11 22:52:28.270453 ip-10-0-138-236 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 22:52:28.270453 ip-10-0-138-236 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 22:52:28.270453 ip-10-0-138-236 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 22:52:28.270453 ip-10-0-138-236 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 22:52:28.270453 ip-10-0-138-236 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 22:52:28.272404 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.272315 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 22:52:28.276412 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276396 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 22:52:28.276412 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276413 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276416 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276420 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276423 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276426 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276429 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276432 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276435 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276438 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276440 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276443 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276446 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276448 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276451 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276458 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276461 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276464 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276467 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276469 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276472 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 22:52:28.276474 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276475 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276478 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276481 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276484 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276487 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276490 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276492 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276495 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276497 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276500 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276503 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276505 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276508 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276510 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276513 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276515 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276518 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276521 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276523 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276526 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 22:52:28.276960 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276528 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276531 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276533 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276536 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276538 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276541 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276546 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276550 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276553 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276555 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276559 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276562 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276565 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276568 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276571 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276573 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276576 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276579 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276582 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276584 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 22:52:28.277443 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276587 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276589 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276592 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276595 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276597 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276600 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276602 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276605 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276608 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276611 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276617 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276620 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276623 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276626 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276628 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276631 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276634 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276636 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276638 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 22:52:28.277975 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276641 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276644 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276646 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276650 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276652 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276655 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276658 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276661 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276663 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276666 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276668 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276671 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276673 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276676 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276678 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276681 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276684 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276687 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.276690 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278078 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 22:52:28.278424 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278086 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278090 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278095 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278097 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278100 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278103 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278106 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278108 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278111 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278114 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278116 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278119 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278122 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278125 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278127 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278130 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278133 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278136 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278139 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278142 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 22:52:28.278903 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278144 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278147 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278150 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278152 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278155 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278157 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278160 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278162 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278165 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278167 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278170 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278172 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278175 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278178 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278180 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278183 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278186 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278188 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278190 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278193 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 22:52:28.279435 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278195 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278198 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278201 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278203 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278205 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278208 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278211 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278214 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278216 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278219 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278222 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278224 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278227 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278229 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278232 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278234 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278237 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278242 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278246 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278249 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 22:52:28.279978 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278252 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278255 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278257 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278260 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278263 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278266 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278268 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278271 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278275 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278279 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278282 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278285 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278288 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278290 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278293 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278295 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278297 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278301 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278304 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 22:52:28.280472 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278307 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278309 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278312 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278316 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278318 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278321 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278324 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278326 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278329 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278331 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278334 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278336 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278339 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278341 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278344 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278347 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278351 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278354 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278357 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 22:52:28.280952 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278722 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278727 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278735 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278738 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278741 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278744 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278747 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278750 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278753 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278755 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278758 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278760 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278763 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278765 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278768 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278770 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278773 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278776 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278779 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 22:52:28.281404 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278781 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278784 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278786 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278805 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278809 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278812 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278815 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278818 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278822 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278825 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278827 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278830 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278833 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278835 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278838 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278856 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278861 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278865 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278868 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278871 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 22:52:28.281979 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278874 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278877 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278880 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278883 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278885 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278888 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278890 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278893 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278895 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278898 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278901 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278903 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278906 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278909 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278911 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278914 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278917 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278920 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278922 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278925 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 22:52:28.282463 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278932 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278935 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278937 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278940 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278942 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278945 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278948 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278950 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278953 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278956 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278958 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278961 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278963 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278966 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278968 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278971 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278974 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278976 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278979 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 22:52:28.282966 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278981 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278984 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278986 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278989 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278991 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278994 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278996 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.278999 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279003 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279006 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279009 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279012 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279015 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279020 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279023 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279025 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279027 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279030 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279033 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279036 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 22:52:28.283427 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279038 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279102 2562 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279114 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279127 2562 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279131 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279135 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279138 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279142 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279147 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279150 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279153 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279156 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279160 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279163 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279166 2562 flags.go:64] FLAG: --cgroup-root="" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279169 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279172 2562 flags.go:64] FLAG: --client-ca-file="" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279175 2562 flags.go:64] FLAG: --cloud-provider="external" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279178 2562 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279182 2562 flags.go:64] FLAG: --cluster-domain="" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279185 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279189 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279192 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279195 2562 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 22:52:28.283920 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279199 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279203 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279206 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279209 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279213 2562 flags.go:64] FLAG: --contention-profiling="false" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279215 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279219 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279222 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279225 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279229 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279232 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279235 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279239 2562 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279242 2562 flags.go:64] FLAG: --enable-server="true" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279245 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279249 2562 flags.go:64] FLAG: --event-burst="100" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279252 2562 flags.go:64] FLAG: --event-qps="50" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279255 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279258 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279261 2562 flags.go:64] FLAG: --eviction-hard="" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279264 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279267 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279270 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279273 2562 flags.go:64] FLAG: --eviction-soft="" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279276 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 22:52:28.284502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279278 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279281 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279284 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279287 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279290 2562 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279293 2562 flags.go:64] FLAG: --feature-gates="" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279297 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279300 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279304 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279307 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279311 2562 flags.go:64] FLAG: --healthz-port="10248" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279314 2562 flags.go:64] FLAG: --help="false" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279317 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279320 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279324 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279327 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279331 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279334 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279337 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279340 2562 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279344 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279347 2562 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279350 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279353 2562 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 22:52:28.285151 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279356 2562 flags.go:64] FLAG: --kube-reserved="" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279359 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279362 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279365 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279367 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279370 2562 flags.go:64] FLAG: --lock-file="" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279373 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279376 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279379 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279384 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279387 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279390 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279394 2562 flags.go:64] FLAG: --logging-format="text" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279396 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279400 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279403 2562 flags.go:64] FLAG: --manifest-url="" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279407 2562 flags.go:64] FLAG: --manifest-url-header="" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279411 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279414 2562 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279418 2562 flags.go:64] FLAG: --max-pods="110" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279421 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279424 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279432 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279435 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279438 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 22:52:28.285740 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279441 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279444 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279451 2562 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279454 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279458 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279461 2562 flags.go:64] FLAG: --pod-cidr="" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279464 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279469 2562 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279472 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279475 2562 flags.go:64] FLAG: --pods-per-core="0" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279478 2562 flags.go:64] FLAG: --port="10250" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279481 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279484 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b11ad8bcffcb226b" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279487 2562 flags.go:64] FLAG: --qos-reserved="" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279490 2562 flags.go:64] FLAG: --read-only-port="10255" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279493 2562 flags.go:64] FLAG: --register-node="true" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279496 2562 flags.go:64] FLAG: --register-with-taints="" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279499 2562 flags.go:64] FLAG: --registry-burst="10" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279502 2562 flags.go:64] FLAG: --registry-qps="5" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279505 2562 flags.go:64] FLAG: --reserved-cpus="" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279508 2562 flags.go:64] FLAG: --reserved-memory="" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279512 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279515 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279517 2562 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279522 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 22:52:28.286371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279525 2562 flags.go:64] FLAG: --runonce="false" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279528 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279531 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279534 2562 flags.go:64] FLAG: --seccomp-default="false" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279538 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279541 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279544 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279547 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279550 2562 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279553 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279556 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279559 2562 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279562 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279566 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279569 2562 flags.go:64] FLAG: --system-cgroups="" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279572 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279577 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279580 2562 flags.go:64] FLAG: --tls-cert-file="" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279583 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279587 2562 flags.go:64] FLAG: --tls-min-version="" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279590 2562 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279593 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279596 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279599 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279602 2562 flags.go:64] FLAG: --v="2" Jun 11 22:52:28.286987 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279606 2562 flags.go:64] FLAG: --version="false" Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279610 2562 flags.go:64] FLAG: --vmodule="" Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279613 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.279616 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279734 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279738 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279743 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279746 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279749 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279753 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279755 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279760 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279763 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279765 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279768 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279771 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279773 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279776 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279778 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279781 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 22:52:28.287582 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279785 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279787 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279790 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279793 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279796 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279798 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279801 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279803 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279806 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279809 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279811 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279814 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279816 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279819 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279822 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279825 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279827 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279830 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279835 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 22:52:28.288080 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279839 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279854 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279858 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279862 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279866 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279869 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279872 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279875 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279877 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279880 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279883 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279886 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279888 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279891 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279899 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279902 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279905 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279907 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279910 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 22:52:28.288571 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279912 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279915 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279917 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279920 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279922 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279925 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279927 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279930 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279933 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279935 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279938 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279940 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279944 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279947 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279973 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.279995 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280000 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280004 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280008 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280011 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 22:52:28.289372 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280014 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280016 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280020 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280023 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280025 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280028 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280030 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280033 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280036 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280039 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280041 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280044 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280047 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280049 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280052 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280055 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280057 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280060 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280062 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280065 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 22:52:28.290001 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280067 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280070 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280072 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280075 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.280079 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.280699 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.287960 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.287979 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.287998 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.288006 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288077 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288082 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288086 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288089 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288092 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 22:52:28.290487 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288095 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288097 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288108 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288111 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288114 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288117 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288119 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288122 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288125 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288127 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288129 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288132 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288134 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288137 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288140 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288142 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288145 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288148 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288150 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 22:52:28.290871 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288153 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288157 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288161 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288163 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288166 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288176 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288179 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288181 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288184 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288187 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288189 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288192 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288194 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288197 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288200 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288202 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288205 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288207 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288210 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 22:52:28.291337 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288212 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288215 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288217 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288220 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288222 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288224 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288227 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288230 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288232 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288235 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288238 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288240 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288242 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288245 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288247 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288250 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288252 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288255 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288258 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288267 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 22:52:28.291798 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288269 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288272 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288275 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288278 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288280 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288282 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288285 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288288 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288290 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288293 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288295 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288298 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288300 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288303 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288305 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288308 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288312 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288315 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288318 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288320 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 22:52:28.292295 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288323 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288325 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288328 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288331 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288333 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288337 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288341 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288343 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288346 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288349 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288353 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288356 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288364 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288367 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288369 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 22:52:28.292770 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.288372 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.288377 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289268 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289284 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289316 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289321 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289326 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289330 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289345 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289349 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289353 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289358 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289362 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289371 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289377 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 22:52:28.293139 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289381 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289386 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289390 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289395 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289399 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289403 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289407 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289412 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289416 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289420 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289429 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289433 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289437 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289442 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289447 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289479 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289483 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289488 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289492 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289497 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 22:52:28.293514 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289501 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289506 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289510 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289519 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289523 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289527 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289531 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289536 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289541 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289545 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289549 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289554 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289558 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289562 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289566 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289575 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289579 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289583 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289588 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 22:52:28.294002 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289592 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289596 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289601 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289605 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289609 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289614 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289619 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289623 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289627 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289664 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289884 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289894 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289897 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289900 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289905 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289909 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289912 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289915 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289918 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289921 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 22:52:28.294465 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289924 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289926 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289929 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289932 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289934 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289938 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289941 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289944 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289947 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289951 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289954 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289957 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289960 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289963 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289966 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289968 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289971 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289974 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289977 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 22:52:28.294957 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289980 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289982 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289985 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289987 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289990 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289993 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289996 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:28.289999 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.290013 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.290148 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.292164 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.293303 2562 server.go:1019] "Starting client certificate rotation" Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.293397 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.293429 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 22:52:28.295400 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.295379 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 22:52:28.297975 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.297960 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 22:52:28.307278 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.307262 2562 log.go:25] "Validated CRI v1 runtime API" Jun 11 22:52:28.310216 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.310195 2562 log.go:25] "Validated CRI v1 image API" Jun 11 22:52:28.311584 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.311561 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 22:52:28.314338 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.314319 2562 fs.go:135] Filesystem UUIDs: map[2b8543a8-135f-4b51-bd46-24272fbf285e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9bcf19e4-94c4-4519-9c47-287b767e503c:/dev/nvme0n1p3] Jun 11 22:52:28.314391 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.314337 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 22:52:28.320482 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.320459 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 22:52:28.320751 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.320639 2562 manager.go:216] Machine: {Timestamp:2026-06-11 22:52:28.319459772 +0000 UTC m=+0.264708655 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099943 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d2e59eb4563210bea030045898d8d SystemUUID:ec2d2e59-eb45-6321-0bea-030045898d8d BootID:f2044582-d613-4215-8b44-262addebace4 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:23:9d:f8:29:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:23:9d:f8:29:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:d8:7b:a0:f1:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 22:52:28.320751 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.320752 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 22:52:28.320887 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.320875 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 22:52:28.323286 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.323255 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 22:52:28.323425 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.323288 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 22:52:28.324486 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.324475 2562 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 22:52:28.324519 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.324489 2562 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 22:52:28.324519 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.324496 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 22:52:28.325051 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.325041 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 22:52:28.326225 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.326216 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 22:52:28.328883 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.328871 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 11 22:52:28.329157 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329147 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 22:52:28.329193 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329178 2562 kubelet.go:486] "Attempting to sync node with API server" Jun 11 22:52:28.329193 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329188 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 22:52:28.329249 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329201 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 22:52:28.329249 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329210 2562 kubelet.go:398] "Adding apiserver pod source" Jun 11 22:52:28.329249 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329222 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 22:52:28.329660 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329642 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 22:52:28.329660 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.329661 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 22:52:28.333331 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.333304 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 22:52:28.334632 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334617 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 22:52:28.334690 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334648 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 22:52:28.334818 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334805 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 22:52:28.334868 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334827 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 22:52:28.334868 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334838 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 22:52:28.334868 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334866 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 22:52:28.334964 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334882 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 22:52:28.334964 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.334894 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 22:52:28.335237 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335225 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 22:52:28.335272 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335238 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 22:52:28.335272 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335246 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 22:52:28.335272 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335251 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 22:52:28.335272 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335260 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 22:52:28.335272 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335268 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 22:52:28.335938 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335928 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 22:52:28.335980 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.335939 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 22:52:28.339149 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.339130 2562 server.go:1268] "Started kubelet" Jun 11 22:52:28.339250 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.339186 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 22:52:28.339250 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.339211 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 22:52:28.339345 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.339308 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 22:52:28.339984 ip-10-0-138-236 systemd[1]: Started Kubernetes Kubelet. Jun 11 22:52:28.341092 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.341076 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 22:52:28.341649 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.341634 2562 server.go:310] "Adding debug handlers to kubelet server" Jun 11 22:52:28.345466 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.345448 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 22:52:28.345549 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.345469 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 22:52:28.345603 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.345561 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.345662 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.345604 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 22:52:28.345751 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.345718 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 22:52:28.345896 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.345876 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 22:52:28.346066 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346050 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 22:52:28.346213 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.345332 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-236.ec2.internal.18b828a688290cf0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-236.ec2.internal,UID:ip-10-0-138-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-236.ec2.internal,},FirstTimestamp:2026-06-11 22:52:28.339105008 +0000 UTC m=+0.284353890,LastTimestamp:2026-06-11 22:52:28.339105008 +0000 UTC m=+0.284353890,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-236.ec2.internal,}" Jun 11 22:52:28.346409 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346392 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 22:52:28.346463 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346416 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 22:52:28.346994 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346573 2562 reconstruct.go:97] "Volume reconstruction finished" Jun 11 22:52:28.346994 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346591 2562 reconciler.go:29] "Reconciler: start to sync state" Jun 11 22:52:28.346994 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346733 2562 factory.go:55] Registering systemd factory Jun 11 22:52:28.346994 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.346750 2562 factory.go:223] Registration of the systemd container factory successfully Jun 11 22:52:28.348261 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.347376 2562 factory.go:153] Registering CRI-O factory Jun 11 22:52:28.348261 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.347392 2562 factory.go:223] Registration of the crio container factory successfully Jun 11 22:52:28.348261 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.347442 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 22:52:28.348261 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.347538 2562 factory.go:103] Registering Raw factory Jun 11 22:52:28.348261 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.347552 2562 manager.go:1196] Started watching for new ooms in manager Jun 11 22:52:28.348261 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.348029 2562 manager.go:349] Starting recovery of all containers Jun 11 22:52:28.355597 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.355421 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 22:52:28.355680 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.355615 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 22:52:28.357796 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.357736 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mxh2f" Jun 11 22:52:28.361695 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.361679 2562 manager.go:354] Recovery completed Jun 11 22:52:28.365763 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.365748 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mxh2f" Jun 11 22:52:28.365811 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.365770 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.367607 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.367591 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.367697 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.367622 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.367697 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.367634 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.368222 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368206 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 22:52:28.368222 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368219 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.368338 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368221 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 22:52:28.368338 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368261 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 11 22:52:28.368940 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368924 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.369015 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368957 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.369015 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.368969 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.369304 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.369290 2562 policy_none.go:49] "None policy: Start" Jun 11 22:52:28.369355 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.369309 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 22:52:28.369355 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.369324 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 22:52:28.370258 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.370247 2562 policy_none.go:47] "Start" Jun 11 22:52:28.406107 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.405694 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 22:52:28.406822 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.406803 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 22:52:28.406889 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.406870 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 22:52:28.406930 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.406919 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 22:52:28.406984 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.406965 2562 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jun 11 22:52:28.408441 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.408424 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 22:52:28.410442 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.410428 2562 manager.go:337] "Starting Device Plugin manager" Jun 11 22:52:28.410534 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.410459 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 22:52:28.410534 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.410471 2562 server.go:87] "Starting device plugin registration server" Jun 11 22:52:28.410757 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.410745 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 22:52:28.410790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.410761 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 22:52:28.410955 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.410939 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 22:52:28.411147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.411069 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 22:52:28.411147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.411088 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 22:52:28.411547 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.411533 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 22:52:28.411593 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.411583 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.508083 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.508038 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal"] Jun 11 22:52:28.508258 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.508166 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.509390 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.509369 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.509520 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.509408 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.509520 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.509423 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.510689 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.510672 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.510803 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.510764 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.510803 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.510802 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.510919 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.510822 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.511484 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511462 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511490 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511492 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511495 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511517 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511536 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511542 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.511572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511568 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.511781 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511511 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.511781 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.511645 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.512814 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.512801 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.512878 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.512825 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 22:52:28.513534 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.513517 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientMemory" Jun 11 22:52:28.513609 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.513545 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 22:52:28.513609 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.513559 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeHasSufficientPID" Jun 11 22:52:28.518509 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.518485 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.518598 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.518508 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-236.ec2.internal\": node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.529822 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.529775 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.536633 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.536613 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-236.ec2.internal\" not found" node="ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.540978 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.540961 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-236.ec2.internal\" not found" node="ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.630817 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.630789 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.647271 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.647243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/934553ba9ec753a59ba628188e42358f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal\" (UID: \"934553ba9ec753a59ba628188e42358f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.647339 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.647280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/934553ba9ec753a59ba628188e42358f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal\" (UID: \"934553ba9ec753a59ba628188e42358f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.647339 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.647312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3cfdc7ebe480fabb219cc62ed7af9efa-config\") pod \"kube-apiserver-proxy-ip-10-0-138-236.ec2.internal\" (UID: \"3cfdc7ebe480fabb219cc62ed7af9efa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.731421 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.731391 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.747554 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.747532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/934553ba9ec753a59ba628188e42358f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal\" (UID: \"934553ba9ec753a59ba628188e42358f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.747618 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.747560 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/934553ba9ec753a59ba628188e42358f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal\" (UID: \"934553ba9ec753a59ba628188e42358f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.747618 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.747589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3cfdc7ebe480fabb219cc62ed7af9efa-config\") pod \"kube-apiserver-proxy-ip-10-0-138-236.ec2.internal\" (UID: \"3cfdc7ebe480fabb219cc62ed7af9efa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.747682 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.747619 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/934553ba9ec753a59ba628188e42358f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal\" (UID: \"934553ba9ec753a59ba628188e42358f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.747682 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.747634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/934553ba9ec753a59ba628188e42358f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal\" (UID: \"934553ba9ec753a59ba628188e42358f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.747682 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.747653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3cfdc7ebe480fabb219cc62ed7af9efa-config\") pod \"kube-apiserver-proxy-ip-10-0-138-236.ec2.internal\" (UID: \"3cfdc7ebe480fabb219cc62ed7af9efa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.831962 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.831883 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:28.839036 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.839021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.844216 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:28.844198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" Jun 11 22:52:28.932681 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:28.932654 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:29.033197 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:29.033157 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:29.133692 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:29.133617 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:29.234134 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:29.234102 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:29.275100 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:29.275069 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934553ba9ec753a59ba628188e42358f.slice/crio-b39c47e6fdbe81e9b71ea5f38862950525d5558e795908dda7d2d474a777f40b WatchSource:0}: Error finding container b39c47e6fdbe81e9b71ea5f38862950525d5558e795908dda7d2d474a777f40b: Status 404 returned error can't find the container with id b39c47e6fdbe81e9b71ea5f38862950525d5558e795908dda7d2d474a777f40b Jun 11 22:52:29.275705 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:29.275267 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfdc7ebe480fabb219cc62ed7af9efa.slice/crio-15e7ddc1022a921cc9beedad8cb004365ce75c75453b11f88ae3f09359a9c506 WatchSource:0}: Error finding container 15e7ddc1022a921cc9beedad8cb004365ce75c75453b11f88ae3f09359a9c506: Status 404 returned error can't find the container with id 15e7ddc1022a921cc9beedad8cb004365ce75c75453b11f88ae3f09359a9c506 Jun 11 22:52:29.278785 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.278769 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 22:52:29.293068 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.293046 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 22:52:29.293572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.293159 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 22:52:29.334528 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:29.334496 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:29.370096 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.370064 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 22:47:28 +0000 UTC" deadline="2028-01-14 08:47:44.044242512 +0000 UTC" Jun 11 22:52:29.370096 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.370089 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13953h55m14.674156598s" Jun 11 22:52:29.410532 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.410444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" event={"ID":"934553ba9ec753a59ba628188e42358f","Type":"ContainerStarted","Data":"b39c47e6fdbe81e9b71ea5f38862950525d5558e795908dda7d2d474a777f40b"} Jun 11 22:52:29.411506 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.411485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" event={"ID":"3cfdc7ebe480fabb219cc62ed7af9efa","Type":"ContainerStarted","Data":"15e7ddc1022a921cc9beedad8cb004365ce75c75453b11f88ae3f09359a9c506"} Jun 11 22:52:29.434626 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:29.434609 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-236.ec2.internal\" not found" Jun 11 22:52:29.487766 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.487746 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 22:52:29.545905 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.545877 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" Jun 11 22:52:29.558935 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.558916 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 22:52:29.560366 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.560354 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" Jun 11 22:52:29.572868 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.571551 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 22:52:29.663376 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.663305 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 22:52:29.829759 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:29.829729 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 22:52:30.264934 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.264906 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 22:52:30.330786 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.330756 2562 apiserver.go:52] "Watching apiserver" Jun 11 22:52:30.342183 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.342151 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 22:52:30.342563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.342536 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-ghcrg","openshift-image-registry/node-ca-p2zlk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal","openshift-multus/multus-4qvkk","openshift-network-diagnostics/network-check-target-lhk4f","openshift-ovn-kubernetes/ovnkube-node-bzc5s","kube-system/global-pull-secret-syncer-cbjj6","kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp","openshift-cluster-node-tuning-operator/tuned-xx7jw","openshift-dns/node-resolver-nmkb6","openshift-multus/multus-additional-cni-plugins-vcsrn","openshift-multus/network-metrics-daemon-kq7cx","openshift-network-operator/iptables-alerter-g6s4n"] Jun 11 22:52:30.343899 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.343875 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.344010 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.343943 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:30.344892 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.344874 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.345005 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.344944 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:30.345909 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.345885 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 22:52:30.346013 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.345895 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.347009 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.346991 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.348143 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.348118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:30.348226 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.348200 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:30.349284 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.349264 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.350530 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.350509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.351763 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.351742 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.352008 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.351987 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.352248 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.352229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 22:52:30.352788 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.352769 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.352922 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.352825 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 22:52:30.353138 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.353121 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.353287 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.353269 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4qsxd\"" Jun 11 22:52:30.353331 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.353312 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 22:52:30.354040 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.353798 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 22:52:30.354600 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.354579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.355982 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.355958 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.356359 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-system-cni-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356447 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-socket-dir-parent\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356447 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.356554 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7jnk\" (UniqueName: \"kubernetes.io/projected/db6bbb27-ddf3-492d-96c9-9681525ea2ae-kube-api-access-q7jnk\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.356554 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn2fp\" (UniqueName: \"kubernetes.io/projected/536c3576-76d0-408b-ae56-f54afb4dd9a8-kube-api-access-nn2fp\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.356554 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356520 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-cni-multus\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356554 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-daemon-config\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-multus-certs\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-etc-kubernetes\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/536c3576-76d0-408b-ae56-f54afb4dd9a8-host-slash\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-cni-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356671 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nxbpc\"" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-cnibin\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356689 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.356744 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-cni-binary-copy\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-k8s-cni-cncf-io\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-kubelet-config\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-dbus\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-os-release\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-netns\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-cni-bin\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-kubelet\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-hostroot\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.356992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-conf-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.357007 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp8dc\" (UniqueName: \"kubernetes.io/projected/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-kube-api-access-cp8dc\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.357131 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.357032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/536c3576-76d0-408b-ae56-f54afb4dd9a8-iptables-alerter-script\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.357995 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.357839 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.360911 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.360894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.362314 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362294 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 22:52:30.362449 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362429 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.362449 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tqwqf\"" Jun 11 22:52:30.362570 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362476 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 22:52:30.362739 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362720 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 22:52:30.362813 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362720 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.362813 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.362750 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 22:52:30.363023 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363007 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 22:52:30.363062 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363053 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.363114 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7xp48\"" Jun 11 22:52:30.363150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363015 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.363180 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363156 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.363274 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.363503 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363486 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lnr7q\"" Jun 11 22:52:30.363577 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.363485 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nvwv5\"" Jun 11 22:52:30.364052 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.364023 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8d6nz\"" Jun 11 22:52:30.364052 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.364043 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 22:52:30.364175 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.364030 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 22:52:30.367062 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.367043 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.367143 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.367105 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 22:52:30.371067 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.371041 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.371591 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.371574 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 22:52:30.371704 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.371621 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bqpvc\"" Jun 11 22:52:30.371704 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.371699 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-855kt\"" Jun 11 22:52:30.371862 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.371714 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 22:52:30.372079 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.372060 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 22:52:30.399014 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.398989 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 22:52:30.414629 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.414612 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-78wld" Jun 11 22:52:30.422258 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.422238 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-78wld" Jun 11 22:52:30.447739 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.447719 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 22:52:30.457250 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-cnibin\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.457385 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:30.457385 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-env-overrides\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.457385 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-serviceca\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn2fp\" (UniqueName: \"kubernetes.io/projected/536c3576-76d0-408b-ae56-f54afb4dd9a8-kube-api-access-nn2fp\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-multus-certs\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-kubelet-config\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-kubernetes\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457506 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-multus-certs\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-cnibin\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-kubelet-config\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.457535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-cni-binary-copy\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-netns\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457583 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-cnibin\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-hostroot\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-conf-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysconfig\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-netns\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457660 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-var-lib-kubelet\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457668 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-hostroot\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457689 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-conf-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/112b0e66-260a-4636-b8ce-427c6c682dab-tmp\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2jvm\" (UniqueName: \"kubernetes.io/projected/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-kube-api-access-t2jvm\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/536c3576-76d0-408b-ae56-f54afb4dd9a8-iptables-alerter-script\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-run\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffw5t\" (UniqueName: \"kubernetes.io/projected/112b0e66-260a-4636-b8ce-427c6c682dab-kube-api-access-ffw5t\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458001 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7jnk\" (UniqueName: \"kubernetes.io/projected/db6bbb27-ddf3-492d-96c9-9681525ea2ae-kube-api-access-q7jnk\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b924723d-0016-433e-8143-dfa369f2c9d7-agent-certs\") pod \"konnectivity-agent-ghcrg\" (UID: \"b924723d-0016-433e-8143-dfa369f2c9d7\") " pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysctl-d\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.457987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-systemd\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-registration-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458059 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-cni-binary-copy\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b924723d-0016-433e-8143-dfa369f2c9d7-konnectivity-ca\") pod \"konnectivity-agent-ghcrg\" (UID: \"b924723d-0016-433e-8143-dfa369f2c9d7\") " pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/112b0e66-260a-4636-b8ce-427c6c682dab-etc-tuned\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-cni-netd\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtxds\" (UniqueName: \"kubernetes.io/projected/149874b1-5504-4544-a52f-c69bf66ee3b3-kube-api-access-wtxds\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-cni-bin\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp8dc\" (UniqueName: \"kubernetes.io/projected/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-kube-api-access-cp8dc\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458242 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-cni-bin\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-etc-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.458266 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-node-log\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.458806 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-device-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/536c3576-76d0-408b-ae56-f54afb4dd9a8-iptables-alerter-script\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.458344 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:52:30.958324234 +0000 UTC m=+2.903573125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458435 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-sys-fs\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458496 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysctl-conf\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-systemd\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-cni-binary-copy\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-log-socket\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovnkube-config\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-cni-multus\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458671 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-cni-multus\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-daemon-config\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-dbus\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.459683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-modprobe-d\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-sys\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-slash\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458882 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-dbus\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458885 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458909 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2vbp\" (UniqueName: \"kubernetes.io/projected/392a9947-87b9-4069-87b1-537d34a59bf3-kube-api-access-v2vbp\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-run-netns\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ckp7\" (UniqueName: \"kubernetes.io/projected/69ae7348-3913-4a9f-a89e-d75b3b3899f7-kube-api-access-2ckp7\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.458987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-os-release\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-system-cni-dir\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459075 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-os-release\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459076 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-os-release\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-host\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-daemon-config\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459137 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-host\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlpjz\" (UniqueName: \"kubernetes.io/projected/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-kube-api-access-hlpjz\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-system-cni-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.460457 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-socket-dir-parent\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459269 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-system-cni-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-systemd-units\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-ovn\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovn-node-metrics-cert\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-socket-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-etc-kubernetes\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459393 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-socket-dir-parent\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/536c3576-76d0-408b-ae56-f54afb4dd9a8-host-slash\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459472 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-etc-kubernetes\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-cni-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/536c3576-76d0-408b-ae56-f54afb4dd9a8-host-slash\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-k8s-cni-cncf-io\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459551 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-multus-cni-dir\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovnkube-script-lib\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149874b1-5504-4544-a52f-c69bf66ee3b3-tmp-dir\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-run-k8s-cni-cncf-io\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.461292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-kubelet\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-host-var-lib-kubelet\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-lib-modules\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-kubelet\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-var-lib-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-cni-bin\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/149874b1-5504-4544-a52f-c69bf66ee3b3-hosts-file\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.459918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.460018 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:30.462089 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.460076 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:52:30.960060432 +0000 UTC m=+2.905309323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:30.463658 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.463631 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 22:52:30.466941 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.466920 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7jnk\" (UniqueName: \"kubernetes.io/projected/db6bbb27-ddf3-492d-96c9-9681525ea2ae-kube-api-access-q7jnk\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.467052 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.466941 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp8dc\" (UniqueName: \"kubernetes.io/projected/2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e-kube-api-access-cp8dc\") pod \"multus-4qvkk\" (UID: \"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e\") " pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.467052 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.466955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn2fp\" (UniqueName: \"kubernetes.io/projected/536c3576-76d0-408b-ae56-f54afb4dd9a8-kube-api-access-nn2fp\") pod \"iptables-alerter-g6s4n\" (UID: \"536c3576-76d0-408b-ae56-f54afb4dd9a8\") " pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.560612 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b924723d-0016-433e-8143-dfa369f2c9d7-konnectivity-ca\") pod \"konnectivity-agent-ghcrg\" (UID: \"b924723d-0016-433e-8143-dfa369f2c9d7\") " pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.560612 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/112b0e66-260a-4636-b8ce-427c6c682dab-etc-tuned\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.560612 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-cni-netd\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtxds\" (UniqueName: \"kubernetes.io/projected/149874b1-5504-4544-a52f-c69bf66ee3b3-kube-api-access-wtxds\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560687 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-cni-netd\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-etc-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-node-log\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-device-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-sys-fs\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560821 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-etc-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560868 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-node-log\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560919 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-device-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-sys-fs\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysctl-conf\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.560962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-systemd\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.560994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561060 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysctl-conf\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561055 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-log-socket\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovnkube-config\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-systemd\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-modprobe-d\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-sys\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561099 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-log-socket\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561176 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-slash\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561205 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-slash\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561246 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-sys\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2vbp\" (UniqueName: \"kubernetes.io/projected/392a9947-87b9-4069-87b1-537d34a59bf3-kube-api-access-v2vbp\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b924723d-0016-433e-8143-dfa369f2c9d7-konnectivity-ca\") pod \"konnectivity-agent-ghcrg\" (UID: \"b924723d-0016-433e-8143-dfa369f2c9d7\") " pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-run-netns\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561318 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-run-netns\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.561683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561224 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-modprobe-d\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckp7\" (UniqueName: \"kubernetes.io/projected/69ae7348-3913-4a9f-a89e-d75b3b3899f7-kube-api-access-2ckp7\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-system-cni-dir\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-os-release\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561400 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561409 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-system-cni-dir\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-host\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-host\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlpjz\" (UniqueName: \"kubernetes.io/projected/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-kube-api-access-hlpjz\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561475 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-os-release\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-host\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-systemd-units\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561517 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-host\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-ovn\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovn-node-metrics-cert\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561557 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-systemd-units\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.562563 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-run-ovn\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-socket-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovnkube-script-lib\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561630 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovnkube-config\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149874b1-5504-4544-a52f-c69bf66ee3b3-tmp-dir\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-socket-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-lib-modules\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-kubelet\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-var-lib-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-cni-bin\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561809 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-kubelet\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/149874b1-5504-4544-a52f-c69bf66ee3b3-hosts-file\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-cnibin\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/149874b1-5504-4544-a52f-c69bf66ee3b3-tmp-dir\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561938 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-lib-modules\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.563407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561959 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.561983 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-var-lib-openvswitch\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-cni-bin\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/392a9947-87b9-4069-87b1-537d34a59bf3-cnibin\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-env-overrides\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/149874b1-5504-4544-a52f-c69bf66ee3b3-hosts-file\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-serviceca\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-kubernetes\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562109 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovnkube-script-lib\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-cni-binary-copy\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562167 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-kubernetes\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysconfig\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-var-lib-kubelet\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/112b0e66-260a-4636-b8ce-427c6c682dab-tmp\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562235 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysconfig\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2jvm\" (UniqueName: \"kubernetes.io/projected/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-kube-api-access-t2jvm\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562301 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-var-lib-kubelet\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562311 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562359 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-run\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffw5t\" (UniqueName: \"kubernetes.io/projected/112b0e66-260a-4636-b8ce-427c6c682dab-kube-api-access-ffw5t\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b924723d-0016-433e-8143-dfa369f2c9d7-agent-certs\") pod \"konnectivity-agent-ghcrg\" (UID: \"b924723d-0016-433e-8143-dfa369f2c9d7\") " pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562451 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-env-overrides\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysctl-d\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562473 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-run\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-systemd\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-registration-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562551 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-serviceca\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562556 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-sysctl-d\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562603 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/112b0e66-260a-4636-b8ce-427c6c682dab-etc-systemd\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562628 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/69ae7348-3913-4a9f-a89e-d75b3b3899f7-registration-dir\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.564997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.562928 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.565661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.563090 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/392a9947-87b9-4069-87b1-537d34a59bf3-cni-binary-copy\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.565661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.563539 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/112b0e66-260a-4636-b8ce-427c6c682dab-etc-tuned\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.565661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.564726 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-ovn-node-metrics-cert\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.565661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.565013 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/112b0e66-260a-4636-b8ce-427c6c682dab-tmp\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.565661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.565295 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b924723d-0016-433e-8143-dfa369f2c9d7-agent-certs\") pod \"konnectivity-agent-ghcrg\" (UID: \"b924723d-0016-433e-8143-dfa369f2c9d7\") " pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.568360 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.568337 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtxds\" (UniqueName: \"kubernetes.io/projected/149874b1-5504-4544-a52f-c69bf66ee3b3-kube-api-access-wtxds\") pod \"node-resolver-nmkb6\" (UID: \"149874b1-5504-4544-a52f-c69bf66ee3b3\") " pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.569128 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.569105 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2vbp\" (UniqueName: \"kubernetes.io/projected/392a9947-87b9-4069-87b1-537d34a59bf3-kube-api-access-v2vbp\") pod \"multus-additional-cni-plugins-vcsrn\" (UID: \"392a9947-87b9-4069-87b1-537d34a59bf3\") " pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.570486 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.570464 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:52:30.570584 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.570492 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:52:30.570584 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.570520 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:30.570671 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.570586 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:52:31.070568331 +0000 UTC m=+3.015817220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:30.573051 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.573010 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ckp7\" (UniqueName: \"kubernetes.io/projected/69ae7348-3913-4a9f-a89e-d75b3b3899f7-kube-api-access-2ckp7\") pod \"aws-ebs-csi-driver-node-zxbbp\" (UID: \"69ae7348-3913-4a9f-a89e-d75b3b3899f7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.573134 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.573095 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2jvm\" (UniqueName: \"kubernetes.io/projected/a7aaaf97-3eac-44f0-9ee0-a308e0f2e805-kube-api-access-t2jvm\") pod \"ovnkube-node-bzc5s\" (UID: \"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.573755 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.573734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffw5t\" (UniqueName: \"kubernetes.io/projected/112b0e66-260a-4636-b8ce-427c6c682dab-kube-api-access-ffw5t\") pod \"tuned-xx7jw\" (UID: \"112b0e66-260a-4636-b8ce-427c6c682dab\") " pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.573827 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.573773 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlpjz\" (UniqueName: \"kubernetes.io/projected/2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751-kube-api-access-hlpjz\") pod \"node-ca-p2zlk\" (UID: \"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751\") " pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.659280 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.659246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g6s4n" Jun 11 22:52:30.667356 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.667335 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4qvkk" Jun 11 22:52:30.673951 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.673930 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:30.682472 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.682454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" Jun 11 22:52:30.687894 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.687877 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:30.695335 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.695313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p2zlk" Jun 11 22:52:30.703779 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.703763 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" Jun 11 22:52:30.710146 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.710132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nmkb6" Jun 11 22:52:30.716433 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.716418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" Jun 11 22:52:30.965329 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.965252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:30.965329 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:30.965315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:30.965528 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.965397 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:30.965528 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.965426 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:30.965528 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.965467 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:52:31.965446135 +0000 UTC m=+3.910695034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:30.965528 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:30.965484 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:52:31.965476915 +0000 UTC m=+3.910725784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:31.124636 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:31.124605 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536c3576_76d0_408b_ae56_f54afb4dd9a8.slice/crio-7907a05840db522a4178afae937ba2017f14ba79086f3287c40e6f77f626c266 WatchSource:0}: Error finding container 7907a05840db522a4178afae937ba2017f14ba79086f3287c40e6f77f626c266: Status 404 returned error can't find the container with id 7907a05840db522a4178afae937ba2017f14ba79086f3287c40e6f77f626c266 Jun 11 22:52:31.125356 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:31.125333 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112b0e66_260a_4636_b8ce_427c6c682dab.slice/crio-55244dd449fc12f460b163d8c7bc1a6b3189cbc3984d58d74e45a4423ee837b0 WatchSource:0}: Error finding container 55244dd449fc12f460b163d8c7bc1a6b3189cbc3984d58d74e45a4423ee837b0: Status 404 returned error can't find the container with id 55244dd449fc12f460b163d8c7bc1a6b3189cbc3984d58d74e45a4423ee837b0 Jun 11 22:52:31.131313 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:31.131292 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ae7348_3913_4a9f_a89e_d75b3b3899f7.slice/crio-1087da134b59767a9ba19e517832d905ec537a670f07fcb99ca3f458ca27eae2 WatchSource:0}: Error finding container 1087da134b59767a9ba19e517832d905ec537a670f07fcb99ca3f458ca27eae2: Status 404 returned error can't find the container with id 1087da134b59767a9ba19e517832d905ec537a670f07fcb99ca3f458ca27eae2 Jun 11 22:52:31.132339 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:31.132294 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392a9947_87b9_4069_87b1_537d34a59bf3.slice/crio-2c30ab203861c7b9033afea15b6345aed43a5ed45fee376cb03a65aac7427f4f WatchSource:0}: Error finding container 2c30ab203861c7b9033afea15b6345aed43a5ed45fee376cb03a65aac7427f4f: Status 404 returned error can't find the container with id 2c30ab203861c7b9033afea15b6345aed43a5ed45fee376cb03a65aac7427f4f Jun 11 22:52:31.132584 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:31.132563 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c595b8d_0a5e_49c3_bf2f_9fdfba4e2751.slice/crio-bc325968dbd63f1e99d800c2f63dec5d8585ebf7aa390c6a7cba4ad918673fde WatchSource:0}: Error finding container bc325968dbd63f1e99d800c2f63dec5d8585ebf7aa390c6a7cba4ad918673fde: Status 404 returned error can't find the container with id bc325968dbd63f1e99d800c2f63dec5d8585ebf7aa390c6a7cba4ad918673fde Jun 11 22:52:31.166896 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.166749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:31.166976 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.166910 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:52:31.166976 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.166928 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:52:31.166976 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.166940 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:31.167080 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.166985 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:52:32.166970956 +0000 UTC m=+4.112219837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:31.407962 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.407937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:31.408285 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.408036 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:31.416895 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.416870 2562 generic.go:356] "Generic (PLEG): container finished" podID="934553ba9ec753a59ba628188e42358f" containerID="0d0ad78239ba451b167f25f2915ac561c99e44d485ea66aea5c0003a33142c07" exitCode=0 Jun 11 22:52:31.416997 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.416946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" event={"ID":"934553ba9ec753a59ba628188e42358f","Type":"ContainerDied","Data":"0d0ad78239ba451b167f25f2915ac561c99e44d485ea66aea5c0003a33142c07"} Jun 11 22:52:31.418004 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.417971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nmkb6" event={"ID":"149874b1-5504-4544-a52f-c69bf66ee3b3","Type":"ContainerStarted","Data":"ca25f341835d06e6a0176cc56ae0fa13a4ed343af000cfa3bcdbbb07973ca8de"} Jun 11 22:52:31.419081 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.419052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerStarted","Data":"2c30ab203861c7b9033afea15b6345aed43a5ed45fee376cb03a65aac7427f4f"} Jun 11 22:52:31.420490 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.420461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" event={"ID":"69ae7348-3913-4a9f-a89e-d75b3b3899f7","Type":"ContainerStarted","Data":"1087da134b59767a9ba19e517832d905ec537a670f07fcb99ca3f458ca27eae2"} Jun 11 22:52:31.421573 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.421553 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" event={"ID":"112b0e66-260a-4636-b8ce-427c6c682dab","Type":"ContainerStarted","Data":"55244dd449fc12f460b163d8c7bc1a6b3189cbc3984d58d74e45a4423ee837b0"} Jun 11 22:52:31.422662 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.422643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g6s4n" event={"ID":"536c3576-76d0-408b-ae56-f54afb4dd9a8","Type":"ContainerStarted","Data":"7907a05840db522a4178afae937ba2017f14ba79086f3287c40e6f77f626c266"} Jun 11 22:52:31.424121 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.424064 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" event={"ID":"3cfdc7ebe480fabb219cc62ed7af9efa","Type":"ContainerStarted","Data":"1260f97fa4fbec4f0e8acf60694ea681f9cbedd855e8df2726df5ed776078bd6"} Jun 11 22:52:31.424214 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.424114 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 22:47:30 +0000 UTC" deadline="2028-02-29 10:08:16.14014658 +0000 UTC" Jun 11 22:52:31.424214 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.424164 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15059h15m44.71598616s" Jun 11 22:52:31.425225 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.425203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4qvkk" event={"ID":"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e","Type":"ContainerStarted","Data":"e4baec1897b55c1fe0f780c673fd5c871e69554f577a896146f655bb24af4f24"} Jun 11 22:52:31.426222 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.426202 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ghcrg" event={"ID":"b924723d-0016-433e-8143-dfa369f2c9d7","Type":"ContainerStarted","Data":"28f39c166df31d412dfb3a005c34d7ecc05669a4010553447ecdaa3459a3be10"} Jun 11 22:52:31.427160 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.427141 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p2zlk" event={"ID":"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751","Type":"ContainerStarted","Data":"bc325968dbd63f1e99d800c2f63dec5d8585ebf7aa390c6a7cba4ad918673fde"} Jun 11 22:52:31.428343 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.428322 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"b8cfea6e781a0a15866a202d834f176accc6a363623423f5b59346d05b98935d"} Jun 11 22:52:31.452042 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.451988 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-236.ec2.internal" podStartSLOduration=2.451969857 podStartE2EDuration="2.451969857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:31.451205345 +0000 UTC m=+3.396454238" watchObservedRunningTime="2026-06-11 22:52:31.451969857 +0000 UTC m=+3.397218752" Jun 11 22:52:31.974508 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.974418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:31.974508 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:31.974500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:31.974725 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.974642 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:31.974725 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.974709 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:52:33.974690601 +0000 UTC m=+5.919939473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:31.975179 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.975157 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:31.975255 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:31.975221 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:52:33.975203983 +0000 UTC m=+5.920452867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:32.176304 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:32.175600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:32.176304 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:32.175785 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:52:32.176304 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:32.175806 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:52:32.176304 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:32.175820 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:32.176304 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:32.175928 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:52:34.175908524 +0000 UTC m=+6.121157397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:32.410193 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:32.410116 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:32.410607 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:32.410265 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:32.410664 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:32.410640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:32.410793 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:32.410720 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:32.424630 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:32.424595 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 22:47:30 +0000 UTC" deadline="2028-03-10 05:10:24.523455574 +0000 UTC" Jun 11 22:52:32.424630 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:32.424629 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15294h17m52.098831382s" Jun 11 22:52:32.449939 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:32.449899 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" event={"ID":"934553ba9ec753a59ba628188e42358f","Type":"ContainerStarted","Data":"cb1a6eb4f680b79abaa977b90d37f388d47a4aef0ca443cd556ce7b40ff8113f"} Jun 11 22:52:33.407206 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:33.407167 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:33.407401 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:33.407330 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:33.989428 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:33.989389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:33.989988 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:33.989470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:33.989988 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:33.989588 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:33.989988 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:33.989651 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:52:37.989631983 +0000 UTC m=+9.934880857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:33.990166 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:33.990049 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:33.990166 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:33.990099 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:52:37.99008471 +0000 UTC m=+9.935333594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:34.190937 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:34.190890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:34.191141 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:34.191122 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:52:34.191231 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:34.191150 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:52:34.191231 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:34.191163 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:34.191231 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:34.191223 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:52:38.191205997 +0000 UTC m=+10.136454879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:34.407705 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:34.407624 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:34.407882 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:34.407763 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:34.407960 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:34.407880 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:34.408014 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:34.407982 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:35.407344 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:35.407298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:35.407796 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:35.407441 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:36.408552 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:36.407945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:36.408552 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:36.408077 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:36.408552 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:36.407952 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:36.408552 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:36.408201 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:37.407283 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:37.407254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:37.407453 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:37.407366 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:38.020132 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:38.020086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:38.020560 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:38.020186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:38.020560 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.020266 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:38.020560 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.020315 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:38.020560 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.020351 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:52:46.020329524 +0000 UTC m=+17.965578408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:38.020560 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.020372 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:52:46.020362239 +0000 UTC m=+17.965611122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:38.221189 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:38.221153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:38.221367 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.221342 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:52:38.221367 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.221365 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:52:38.221475 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.221379 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:38.221475 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.221450 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:52:46.221430084 +0000 UTC m=+18.166678977 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:38.409312 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:38.408807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:38.409312 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.408933 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:38.409312 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:38.409221 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:38.409573 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:38.409349 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:39.407366 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:39.407328 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:39.407798 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:39.407479 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:40.408505 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:40.408190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:40.408505 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:40.408326 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:40.408505 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:40.408374 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:40.409033 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:40.408536 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:41.407734 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:41.407697 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:41.407920 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:41.407812 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:42.407776 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:42.407733 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:42.408239 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:42.407887 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:42.408239 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:42.407951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:42.408239 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:42.408081 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:43.407281 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:43.407249 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:43.407469 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:43.407380 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:44.407700 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:44.407671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:44.408209 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:44.407800 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:44.408209 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:44.407893 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:44.408209 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:44.408014 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:45.407918 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:45.407808 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:45.408384 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:45.407963 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:46.086233 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:46.086197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:46.086407 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:46.086260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:46.086407 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.086360 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:46.086407 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.086377 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:46.086522 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.086431 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:53:02.086409686 +0000 UTC m=+34.031658555 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:52:46.086522 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.086451 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:53:02.086443103 +0000 UTC m=+34.031691972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:52:46.287681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:46.287645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:46.287882 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.287840 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:52:46.287882 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.287876 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:52:46.287994 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.287888 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:46.287994 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.287956 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:02.287936927 +0000 UTC m=+34.233185812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:52:46.408175 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:46.408040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:46.408615 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:46.408075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:46.408615 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.408239 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:46.408615 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:46.408329 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:47.408009 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:47.407976 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:47.408185 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:47.408103 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:48.408031 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:48.407985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:48.408206 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:48.408094 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:48.408579 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:48.408473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:48.408657 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:48.408584 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:49.407989 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:49.407967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:49.408103 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:49.408060 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:50.408443 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.407945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:50.408443 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.408005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:50.409273 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:50.408497 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:50.409273 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:50.408584 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:50.493685 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.493660 2562 generic.go:356] "Generic (PLEG): container finished" podID="392a9947-87b9-4069-87b1-537d34a59bf3" containerID="13b4d91cc1d1aefedd4f0a490b856af26b022785c3482db7a15bcacc42dab308" exitCode=0 Jun 11 22:52:50.493828 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.493751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerDied","Data":"13b4d91cc1d1aefedd4f0a490b856af26b022785c3482db7a15bcacc42dab308"} Jun 11 22:52:50.498129 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.495949 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" event={"ID":"69ae7348-3913-4a9f-a89e-d75b3b3899f7","Type":"ContainerStarted","Data":"8fe9d34a530ad15d1943e665b5a3c212e6cfb30c4934a4cb2a9fc56bc2fba4a1"} Jun 11 22:52:50.499308 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.499279 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" event={"ID":"112b0e66-260a-4636-b8ce-427c6c682dab","Type":"ContainerStarted","Data":"a15edf8a5f1652165b387353c235a4b1a45ae6b654612c644c46bd244c4ab447"} Jun 11 22:52:50.500835 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.500815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4qvkk" event={"ID":"2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e","Type":"ContainerStarted","Data":"fe195518ca7d3507491c0096e13f812777d622274970df8088b4af4b57abfb97"} Jun 11 22:52:50.502242 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.502224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ghcrg" event={"ID":"b924723d-0016-433e-8143-dfa369f2c9d7","Type":"ContainerStarted","Data":"06b0906368cc82b7f4e362079b530db3f8a36de7e80348154568445e21670c5e"} Jun 11 22:52:50.503737 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.503714 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p2zlk" event={"ID":"2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751","Type":"ContainerStarted","Data":"f2f7b645e910286c6cbe0e3888f798b5284971a93c2ba424e2b2d24a97301b12"} Jun 11 22:52:50.507859 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.507830 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:52:50.508289 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.508260 2562 generic.go:356] "Generic (PLEG): container finished" podID="a7aaaf97-3eac-44f0-9ee0-a308e0f2e805" containerID="b3e970b3ed5081dbaf3087400483087a1d503277e997edb3b2d88b367cabef8c" exitCode=1 Jun 11 22:52:50.508375 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.508332 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"f441c5567dc9b0466228c172eda11b21d1a7e9df2e7f083eba03dfeac41c2148"} Jun 11 22:52:50.508432 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.508385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"e7a683de8c5f98f1c6bba9ba5b991fbca8867359a8f05084e24ab699ea574793"} Jun 11 22:52:50.508432 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.508425 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerDied","Data":"b3e970b3ed5081dbaf3087400483087a1d503277e997edb3b2d88b367cabef8c"} Jun 11 22:52:50.508531 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.508456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"e0827295ad4a5db45b5c0a5e07a8eef1fbb80a16d11967a8375732e443d96f4c"} Jun 11 22:52:50.509715 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.509693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nmkb6" event={"ID":"149874b1-5504-4544-a52f-c69bf66ee3b3","Type":"ContainerStarted","Data":"82dde99d87f24419eb35363dcca2db44f8c1431253aef7b7d3bdaaf6c105e19a"} Jun 11 22:52:50.516444 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.516410 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-236.ec2.internal" podStartSLOduration=21.516399098 podStartE2EDuration="21.516399098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:32.474224931 +0000 UTC m=+4.419473823" watchObservedRunningTime="2026-06-11 22:52:50.516399098 +0000 UTC m=+22.461647989" Jun 11 22:52:50.530543 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.530499 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nmkb6" podStartSLOduration=4.242597011 podStartE2EDuration="22.530484025s" totalImagesPullingTime="18.287887014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:50.530449836 +0000 UTC m=+22.475698729" watchObservedRunningTime="2026-06-11 22:52:50.530484025 +0000 UTC m=+22.475732914" Jun 11 22:52:50.547492 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.547452 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4qvkk" podStartSLOduration=4.253158656 podStartE2EDuration="22.547440563s" totalImagesPullingTime="18.294281907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:50.546891774 +0000 UTC m=+22.492140681" watchObservedRunningTime="2026-06-11 22:52:50.547440563 +0000 UTC m=+22.492689454" Jun 11 22:52:50.561203 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.561170 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ghcrg" podStartSLOduration=4.350128123 podStartE2EDuration="22.561161234s" totalImagesPullingTime="18.211033111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:50.560963306 +0000 UTC m=+22.506212198" watchObservedRunningTime="2026-06-11 22:52:50.561161234 +0000 UTC m=+22.506410125" Jun 11 22:52:50.575523 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.575488 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p2zlk" podStartSLOduration=4.312614535 podStartE2EDuration="22.575476758s" totalImagesPullingTime="18.262862223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:50.574902242 +0000 UTC m=+22.520151134" watchObservedRunningTime="2026-06-11 22:52:50.575476758 +0000 UTC m=+22.520725653" Jun 11 22:52:50.667526 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.667458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:50.744727 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:50.744693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:51.029240 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.029214 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 22:52:51.407082 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.407056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:51.407302 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:51.407154 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:51.423175 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.423063 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T22:52:51.029235621Z","UUID":"b3bf0d26-dd9a-47b1-a455-e17fa91819cc","Handler":null,"Name":"","Endpoint":""} Jun 11 22:52:51.424427 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.424404 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 22:52:51.424503 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.424438 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 22:52:51.514399 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.514363 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" event={"ID":"69ae7348-3913-4a9f-a89e-d75b3b3899f7","Type":"ContainerStarted","Data":"41c28e4f045a92a9881dc7076d5de23b3f4584cf4b73c0e0f0e44fb2704298d3"} Jun 11 22:52:51.516235 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.516208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g6s4n" event={"ID":"536c3576-76d0-408b-ae56-f54afb4dd9a8","Type":"ContainerStarted","Data":"a0606f177e0c295f48c3f798ca31d3ab3f99a6cd30ecd8fc56c356165c1c66ae"} Jun 11 22:52:51.519982 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.519935 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:52:51.520930 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.520488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"1c1788bc3e18910a3c1fe15365ef4a0d1985000d9f49191e24cbf61c1e7dad13"} Jun 11 22:52:51.520930 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.520654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"a0a3282c1f1ec3f163ca0e05c600e2006a03aa4748f44fd25c5bd100ec81b23f"} Jun 11 22:52:51.533078 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.533013 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xx7jw" podStartSLOduration=5.239110965 podStartE2EDuration="23.533000311s" totalImagesPullingTime="18.293889346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:50.597331768 +0000 UTC m=+22.542580660" watchObservedRunningTime="2026-06-11 22:52:51.533000311 +0000 UTC m=+23.478249202" Jun 11 22:52:51.533371 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.533342 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g6s4n" podStartSLOduration=5.241619575 podStartE2EDuration="23.533333144s" totalImagesPullingTime="18.291713569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:51.53274953 +0000 UTC m=+23.477998422" watchObservedRunningTime="2026-06-11 22:52:51.533333144 +0000 UTC m=+23.478582035" Jun 11 22:52:51.751103 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.751058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:51.861308 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:51.861272 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:52.408015 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:52.407941 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:52.408186 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:52.407942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:52.408186 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:52.408070 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:52.408186 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:52.408120 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:52.525663 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:52.525623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" event={"ID":"69ae7348-3913-4a9f-a89e-d75b3b3899f7","Type":"ContainerStarted","Data":"734356f9ee32a0fca08415ce642d806ba167aab198ce8f8ec3724ae127563a17"} Jun 11 22:52:52.544561 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:52.544513 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zxbbp" podStartSLOduration=4.114016653 podStartE2EDuration="24.544498424s" totalImagesPullingTime="20.430481771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:52.543914872 +0000 UTC m=+24.489163787" watchObservedRunningTime="2026-06-11 22:52:52.544498424 +0000 UTC m=+24.489747314" Jun 11 22:52:53.407615 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:53.407420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:53.407766 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:53.407741 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:53.531622 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:53.531591 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:52:53.532147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:53.532117 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"acd0b51ed9b062f2df691b73e89eec9ba3391b7cff97e55e33189d39fd87ce39"} Jun 11 22:52:53.719902 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:53.719869 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:53.720390 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:53.720368 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:54.407167 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:54.407140 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:54.407323 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:54.407143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:54.407323 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:54.407245 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:54.407434 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:54.407344 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:54.534433 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:54.534392 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:54.534974 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:54.534938 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ghcrg" Jun 11 22:52:54.588907 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:54.588881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:55.407552 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:55.407466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:55.407697 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:55.407602 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:56.408236 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:56.408105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:56.408634 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:56.408154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:56.408634 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:56.408328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:56.408634 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:56.408417 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:56.541258 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:56.541236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:52:56.541551 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:56.541532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"33d3af246e3843240438224bcd099ad8d0174623e2fc3d5e174468a5b8ef9d8a"} Jun 11 22:52:56.542292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:56.542274 2562 scope.go:117] "RemoveContainer" containerID="b3e970b3ed5081dbaf3087400483087a1d503277e997edb3b2d88b367cabef8c" Jun 11 22:52:57.407441 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.407414 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:57.407643 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:57.407512 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:57.546349 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.546257 2562 generic.go:356] "Generic (PLEG): container finished" podID="392a9947-87b9-4069-87b1-537d34a59bf3" containerID="f777333b531a84ad857a76a6e0ba8e91ce4220a81c56b74c4720f78d0b922bf2" exitCode=0 Jun 11 22:52:57.546784 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.546372 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerDied","Data":"f777333b531a84ad857a76a6e0ba8e91ce4220a81c56b74c4720f78d0b922bf2"} Jun 11 22:52:57.561186 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.561160 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:52:57.561547 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.561512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" event={"ID":"a7aaaf97-3eac-44f0-9ee0-a308e0f2e805","Type":"ContainerStarted","Data":"d1d463f3e4686cea64a819d18487eb47e098fdb765642573b771a4038742b59a"} Jun 11 22:52:57.561889 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.561737 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:57.561889 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.561815 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:57.561889 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.561882 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:57.579385 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.579356 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:57.579602 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.579584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:52:57.625804 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.625752 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" podStartSLOduration=10.782719847 podStartE2EDuration="29.625738013s" totalImagesPullingTime="18.843018166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:52:57.624470989 +0000 UTC m=+29.569719881" watchObservedRunningTime="2026-06-11 22:52:57.625738013 +0000 UTC m=+29.570986921" Jun 11 22:52:57.966018 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:57.965983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:58.026865 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.026815 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:58.130214 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.130176 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:58.408494 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.408468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:58.408615 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.408577 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:58.408615 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:58.408588 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:58.408732 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:58.408649 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:58.445654 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:52:58.445633 2562 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392a9947_87b9_4069_87b1_537d34a59bf3.slice/crio-93283fb189544440e38774b7105dcf3faefe29c12396e489f6a16fa76f2beea2.scope/cpu.max": read /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392a9947_87b9_4069_87b1_537d34a59bf3.slice/crio-93283fb189544440e38774b7105dcf3faefe29c12396e489f6a16fa76f2beea2.scope/cpu.max: no such device Jun 11 22:52:58.566409 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.566153 2562 generic.go:356] "Generic (PLEG): container finished" podID="392a9947-87b9-4069-87b1-537d34a59bf3" containerID="93283fb189544440e38774b7105dcf3faefe29c12396e489f6a16fa76f2beea2" exitCode=0 Jun 11 22:52:58.566801 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.566232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerDied","Data":"93283fb189544440e38774b7105dcf3faefe29c12396e489f6a16fa76f2beea2"} Jun 11 22:52:58.622811 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.622732 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbjj6"] Jun 11 22:52:58.622982 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.622869 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:52:58.623044 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:58.622983 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:52:58.626150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.626126 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lhk4f"] Jun 11 22:52:58.626273 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.626224 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:52:58.626341 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:58.626322 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:52:58.626819 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.626798 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kq7cx"] Jun 11 22:52:58.626955 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:58.626899 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:52:58.627016 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:52:58.626981 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:52:59.027906 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:59.027869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:52:59.090382 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:52:59.090345 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:00.407982 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:00.407955 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:00.408656 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:00.407952 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:53:00.408656 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:00.408051 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:53:00.408656 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:00.408139 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:53:00.408656 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:00.407952 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:00.408656 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:00.408206 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:53:00.572958 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:00.572921 2562 generic.go:356] "Generic (PLEG): container finished" podID="392a9947-87b9-4069-87b1-537d34a59bf3" containerID="277e187c71999b5dafa8171091ce440f6158ecc31fc07d43525c526b5db1ba98" exitCode=0 Jun 11 22:53:00.573113 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:00.572964 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerDied","Data":"277e187c71999b5dafa8171091ce440f6158ecc31fc07d43525c526b5db1ba98"} Jun 11 22:53:02.104624 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.104591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:53:02.105136 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.104657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:02.105136 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.104754 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 22:53:02.105136 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.104812 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret podName:3d46664d-dfda-4138-aa94-ff3bb5cf5b8f nodeName:}" failed. No retries permitted until 2026-06-11 22:53:34.104798325 +0000 UTC m=+66.050047194 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret") pod "global-pull-secret-syncer-cbjj6" (UID: "3d46664d-dfda-4138-aa94-ff3bb5cf5b8f") : object "kube-system"/"original-pull-secret" not registered Jun 11 22:53:02.105136 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.104754 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:53:02.105136 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.104941 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:53:34.104918367 +0000 UTC m=+66.050167242 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 22:53:02.305738 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.305710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:02.305924 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.305874 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 22:53:02.305924 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.305901 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 22:53:02.305924 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.305910 2562 projected.go:196] Error preparing data for projected volume kube-api-access-mw5wp for pod openshift-network-diagnostics/network-check-target-lhk4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:53:02.306032 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.305963 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp podName:6a319908-03ee-432c-ab8c-6f76a64df422 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:34.305946999 +0000 UTC m=+66.251195868 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mw5wp" (UniqueName: "kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp") pod "network-check-target-lhk4f" (UID: "6a319908-03ee-432c-ab8c-6f76a64df422") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 22:53:02.407282 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.407197 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:02.407449 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.407197 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:02.407449 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.407340 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:53:02.407449 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.407387 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhk4f" podUID="6a319908-03ee-432c-ab8c-6f76a64df422" Jun 11 22:53:02.407835 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.407805 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:53:02.407928 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:02.407337 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbjj6" podUID="3d46664d-dfda-4138-aa94-ff3bb5cf5b8f" Jun 11 22:53:02.616492 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:02.616459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:03.393508 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.393439 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-236.ec2.internal" event="NodeReady" Jun 11 22:53:03.393901 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.393572 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 22:53:03.443737 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.443709 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q65s9"] Jun 11 22:53:03.447374 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.447358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.450644 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.450410 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9z2n\"" Jun 11 22:53:03.450644 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.450477 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 22:53:03.450644 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.450561 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 22:53:03.450940 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.450700 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fcl8g"] Jun 11 22:53:03.456054 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.455164 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.456054 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.456029 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q65s9"] Jun 11 22:53:03.458350 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.458130 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wbppm\"" Jun 11 22:53:03.458350 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.458146 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 22:53:03.458513 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.458451 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 22:53:03.458728 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.458710 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 22:53:03.462409 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.462386 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fcl8g"] Jun 11 22:53:03.617567 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.617518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjqzx\" (UniqueName: \"kubernetes.io/projected/4643df29-4bfb-4cf1-ba2b-226430b8f802-kube-api-access-jjqzx\") pod \"ingress-canary-fcl8g\" (UID: \"4643df29-4bfb-4cf1-ba2b-226430b8f802\") " pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.617567 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.617573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2hn\" (UniqueName: \"kubernetes.io/projected/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-kube-api-access-wx2hn\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.617799 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.617639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-config-volume\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.617799 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.617671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-tmp-dir\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.617799 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.617786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.617950 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.617834 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4643df29-4bfb-4cf1-ba2b-226430b8f802-cert\") pod \"ingress-canary-fcl8g\" (UID: \"4643df29-4bfb-4cf1-ba2b-226430b8f802\") " pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.718789 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.718748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4643df29-4bfb-4cf1-ba2b-226430b8f802-cert\") pod \"ingress-canary-fcl8g\" (UID: \"4643df29-4bfb-4cf1-ba2b-226430b8f802\") " pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.718996 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.718803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjqzx\" (UniqueName: \"kubernetes.io/projected/4643df29-4bfb-4cf1-ba2b-226430b8f802-kube-api-access-jjqzx\") pod \"ingress-canary-fcl8g\" (UID: \"4643df29-4bfb-4cf1-ba2b-226430b8f802\") " pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.718996 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.718839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2hn\" (UniqueName: \"kubernetes.io/projected/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-kube-api-access-wx2hn\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.718996 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.718946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-config-volume\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.718996 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.718992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-tmp-dir\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.719185 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.719058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.719185 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:03.719154 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:03.719294 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:03.719222 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:04.219202321 +0000 UTC m=+36.164451210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:03.719420 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.719392 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-tmp-dir\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.719575 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.719555 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-config-volume\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.723914 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.723889 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4643df29-4bfb-4cf1-ba2b-226430b8f802-cert\") pod \"ingress-canary-fcl8g\" (UID: \"4643df29-4bfb-4cf1-ba2b-226430b8f802\") " pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.728763 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.728720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjqzx\" (UniqueName: \"kubernetes.io/projected/4643df29-4bfb-4cf1-ba2b-226430b8f802-kube-api-access-jjqzx\") pod \"ingress-canary-fcl8g\" (UID: \"4643df29-4bfb-4cf1-ba2b-226430b8f802\") " pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.729091 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.729070 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2hn\" (UniqueName: \"kubernetes.io/projected/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-kube-api-access-wx2hn\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:03.768148 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.768120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fcl8g" Jun 11 22:53:03.941256 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:03.941209 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:04.007556 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.007461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:04.222331 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.222297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:04.222514 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:04.222430 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:04.222514 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:04.222505 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:05.22248302 +0000 UTC m=+37.167731902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:04.407728 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.407648 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:53:04.408310 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.407646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:04.408310 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.407648 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:04.412367 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.412344 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 22:53:04.412367 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.412361 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 22:53:04.412562 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.412383 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 22:53:04.412562 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.412399 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4mdrn\"" Jun 11 22:53:04.412562 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.412445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s94r6\"" Jun 11 22:53:04.412562 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:04.412525 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 22:53:05.169420 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:05.169393 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fcl8g"] Jun 11 22:53:05.181892 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:53:05.181860 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4643df29_4bfb_4cf1_ba2b_226430b8f802.slice/crio-9bfe760136d667c4b14423dfc034b49f63aae4ea1d8021dce9fa0b3a8b15185f WatchSource:0}: Error finding container 9bfe760136d667c4b14423dfc034b49f63aae4ea1d8021dce9fa0b3a8b15185f: Status 404 returned error can't find the container with id 9bfe760136d667c4b14423dfc034b49f63aae4ea1d8021dce9fa0b3a8b15185f Jun 11 22:53:05.229227 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:05.229203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:05.229358 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:05.229339 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:05.229423 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:05.229412 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:07.229391986 +0000 UTC m=+39.174640872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:05.586516 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:05.586433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fcl8g" event={"ID":"4643df29-4bfb-4cf1-ba2b-226430b8f802","Type":"ContainerStarted","Data":"9bfe760136d667c4b14423dfc034b49f63aae4ea1d8021dce9fa0b3a8b15185f"} Jun 11 22:53:05.589616 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:05.589586 2562 generic.go:356] "Generic (PLEG): container finished" podID="392a9947-87b9-4069-87b1-537d34a59bf3" containerID="2799dc5eb9d192368ca93f55ddf57806262d8dc804ef8b2f2be5e9661cfc47af" exitCode=0 Jun 11 22:53:05.589779 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:05.589629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerDied","Data":"2799dc5eb9d192368ca93f55ddf57806262d8dc804ef8b2f2be5e9661cfc47af"} Jun 11 22:53:06.595501 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:06.595470 2562 generic.go:356] "Generic (PLEG): container finished" podID="392a9947-87b9-4069-87b1-537d34a59bf3" containerID="be165b3b34e32bf9f9eea8d379141d23b344a049c0f4ed7a2a0f364ed1f60cf7" exitCode=0 Jun 11 22:53:06.595983 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:06.595508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerDied","Data":"be165b3b34e32bf9f9eea8d379141d23b344a049c0f4ed7a2a0f364ed1f60cf7"} Jun 11 22:53:07.012460 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.012277 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:07.072238 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.072208 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:07.245685 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.245650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:07.245883 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:07.245756 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:07.245883 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:07.245826 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:11.245805722 +0000 UTC m=+43.191054598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:07.600917 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.600882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" event={"ID":"392a9947-87b9-4069-87b1-537d34a59bf3","Type":"ContainerStarted","Data":"13b13a2a3b52e7da85864e9438c0739faa3a87d4ad31a08f8965abb05ae31179"} Jun 11 22:53:07.602238 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.602211 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fcl8g" event={"ID":"4643df29-4bfb-4cf1-ba2b-226430b8f802","Type":"ContainerStarted","Data":"6b28e745ad981d65b2ffed1988f41fa61dd0ab540a91f1696e09833a31e3ab5c"} Jun 11 22:53:07.627369 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.627323 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vcsrn" podStartSLOduration=8.989950063 podStartE2EDuration="39.627308902s" totalImagesPullingTime="30.637358839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:53:07.625838873 +0000 UTC m=+39.571087764" watchObservedRunningTime="2026-06-11 22:53:07.627308902 +0000 UTC m=+39.572557792" Jun 11 22:53:07.643549 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.643504 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fcl8g" podStartSLOduration=2.757999511 podStartE2EDuration="4.643489838s" totalImagesPullingTime="1.885490327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:53:07.642938005 +0000 UTC m=+39.588186897" watchObservedRunningTime="2026-06-11 22:53:07.643489838 +0000 UTC m=+39.588738729" Jun 11 22:53:07.758913 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.758878 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:07.858011 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:07.857930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:08.905903 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:08.905871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:10.355302 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:10.355267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:10.443748 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:10.443716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:11.274937 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:11.274901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:11.275101 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:11.275019 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:11.275101 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:11.275076 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:19.275060772 +0000 UTC m=+51.220309644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:14.119912 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:14.119875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:14.181898 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:14.181870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:14.593597 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:14.593562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:14.658569 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:14.658529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:19.332009 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:19.331970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:19.332493 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:19.332112 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:19.332493 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:19.332195 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:53:35.332171923 +0000 UTC m=+67.277420791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:27.307415 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.307377 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb"] Jun 11 22:53:27.310201 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.310176 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.313186 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.313165 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 22:53:27.313310 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.313200 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 22:53:27.313926 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.313906 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 22:53:27.314026 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.313931 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 11 22:53:27.314071 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.314053 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd"] Jun 11 22:53:27.315799 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.315783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.317746 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.317730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb"] Jun 11 22:53:27.318437 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.318412 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 11 22:53:27.318437 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.318433 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 11 22:53:27.318604 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.318478 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 11 22:53:27.318604 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.318502 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 11 22:53:27.327557 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.327538 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd"] Jun 11 22:53:27.388408 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388377 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-ca\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.388408 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0335807d-c705-445c-bbe7-63368f2effdd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.388661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-hub\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.388661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.388661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/502f73b6-cc70-49cf-a6de-f94fcb6d084f-klusterlet-config\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.388661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.388661 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q7hb\" (UniqueName: \"kubernetes.io/projected/0335807d-c705-445c-bbe7-63368f2effdd-kube-api-access-6q7hb\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.388871 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/502f73b6-cc70-49cf-a6de-f94fcb6d084f-tmp\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.388871 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.388717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdwrr\" (UniqueName: \"kubernetes.io/projected/502f73b6-cc70-49cf-a6de-f94fcb6d084f-kube-api-access-sdwrr\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.489804 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-ca\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.489804 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0335807d-c705-445c-bbe7-63368f2effdd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-hub\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/502f73b6-cc70-49cf-a6de-f94fcb6d084f-klusterlet-config\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.489983 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q7hb\" (UniqueName: \"kubernetes.io/projected/0335807d-c705-445c-bbe7-63368f2effdd-kube-api-access-6q7hb\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.490009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/502f73b6-cc70-49cf-a6de-f94fcb6d084f-tmp\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.490055 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.490033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwrr\" (UniqueName: \"kubernetes.io/projected/502f73b6-cc70-49cf-a6de-f94fcb6d084f-kube-api-access-sdwrr\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.490583 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.490552 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/502f73b6-cc70-49cf-a6de-f94fcb6d084f-tmp\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.490741 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.490720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0335807d-c705-445c-bbe7-63368f2effdd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.494117 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.494089 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.494117 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.494101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-ca\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.494241 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.494129 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/502f73b6-cc70-49cf-a6de-f94fcb6d084f-klusterlet-config\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.494297 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.494275 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-hub\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.494360 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.494347 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0335807d-c705-445c-bbe7-63368f2effdd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.497995 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.497966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdwrr\" (UniqueName: \"kubernetes.io/projected/502f73b6-cc70-49cf-a6de-f94fcb6d084f-kube-api-access-sdwrr\") pod \"klusterlet-addon-workmgr-6694ddcb6b-6njwb\" (UID: \"502f73b6-cc70-49cf-a6de-f94fcb6d084f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.498084 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.498039 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q7hb\" (UniqueName: \"kubernetes.io/projected/0335807d-c705-445c-bbe7-63368f2effdd-kube-api-access-6q7hb\") pod \"cluster-proxy-proxy-agent-84cd6c48bf-28lgd\" (UID: \"0335807d-c705-445c-bbe7-63368f2effdd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.620535 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.620455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:27.635507 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.635484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:53:27.754495 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.754457 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb"] Jun 11 22:53:27.758315 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:53:27.758286 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502f73b6_cc70_49cf_a6de_f94fcb6d084f.slice/crio-e0f0d84523c91f19b9bd4982ef5a08d0eb3d2270c1873aae1009a45d671183f6 WatchSource:0}: Error finding container e0f0d84523c91f19b9bd4982ef5a08d0eb3d2270c1873aae1009a45d671183f6: Status 404 returned error can't find the container with id e0f0d84523c91f19b9bd4982ef5a08d0eb3d2270c1873aae1009a45d671183f6 Jun 11 22:53:27.774127 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:27.774102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd"] Jun 11 22:53:27.776480 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:53:27.776457 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0335807d_c705_445c_bbe7_63368f2effdd.slice/crio-02110397732e4501391ed6814c3004acd23c7bbcc3a87c3877920af6cb388d68 WatchSource:0}: Error finding container 02110397732e4501391ed6814c3004acd23c7bbcc3a87c3877920af6cb388d68: Status 404 returned error can't find the container with id 02110397732e4501391ed6814c3004acd23c7bbcc3a87c3877920af6cb388d68 Jun 11 22:53:28.653569 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:28.653528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" event={"ID":"0335807d-c705-445c-bbe7-63368f2effdd","Type":"ContainerStarted","Data":"02110397732e4501391ed6814c3004acd23c7bbcc3a87c3877920af6cb388d68"} Jun 11 22:53:28.655120 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:28.655079 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" event={"ID":"502f73b6-cc70-49cf-a6de-f94fcb6d084f","Type":"ContainerStarted","Data":"e0f0d84523c91f19b9bd4982ef5a08d0eb3d2270c1873aae1009a45d671183f6"} Jun 11 22:53:29.585135 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:29.585080 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzc5s" Jun 11 22:53:29.742857 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:29.742717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:29.855383 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:29.855306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:33.672102 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.672055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" event={"ID":"0335807d-c705-445c-bbe7-63368f2effdd","Type":"ContainerStarted","Data":"88078b0874d2d994a992c68a93ef23f7463809068e6d1c818c84d07938398c04"} Jun 11 22:53:33.672622 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.672113 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" event={"ID":"0335807d-c705-445c-bbe7-63368f2effdd","Type":"ContainerStarted","Data":"34b75848c83e1aedc5127155f63a4503ae8d622a54658a9e4228f3b7597701b7"} Jun 11 22:53:33.672622 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.672133 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" event={"ID":"0335807d-c705-445c-bbe7-63368f2effdd","Type":"ContainerStarted","Data":"e48e5411c60c0726891c93104db7b91916b6b5a30ee3dd51c1525c39a1e8b4c0"} Jun 11 22:53:33.673443 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.673423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" event={"ID":"502f73b6-cc70-49cf-a6de-f94fcb6d084f","Type":"ContainerStarted","Data":"ea7622e9857f216871078639f190e30df6a5e425ae683df4953b4fb8e8f904e0"} Jun 11 22:53:33.673589 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.673571 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:33.675251 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.675232 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:53:33.695141 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.695101 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" podStartSLOduration=1.712265143 podStartE2EDuration="6.695089988s" totalImagesPullingTime="4.982824845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:53:33.694627497 +0000 UTC m=+65.639876389" watchObservedRunningTime="2026-06-11 22:53:33.695089988 +0000 UTC m=+65.640338878" Jun 11 22:53:33.722445 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:33.722407 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" podStartSLOduration=1.723096873 podStartE2EDuration="6.72239492s" totalImagesPullingTime="4.999298047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:53:33.722324319 +0000 UTC m=+65.667573227" watchObservedRunningTime="2026-06-11 22:53:33.72239492 +0000 UTC m=+65.667643844" Jun 11 22:53:34.146681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.146628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:53:34.146890 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.146724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:34.149610 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.149592 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 22:53:34.149658 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.149615 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 22:53:34.157784 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:34.157764 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 22:53:34.157854 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:34.157833 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:54:38.157815224 +0000 UTC m=+130.103064093 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : secret "metrics-daemon-secret" not found Jun 11 22:53:34.159866 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.159837 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d46664d-dfda-4138-aa94-ff3bb5cf5b8f-original-pull-secret\") pod \"global-pull-secret-syncer-cbjj6\" (UID: \"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f\") " pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:34.347868 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.347802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:34.351032 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.351015 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 22:53:34.360891 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.360874 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 22:53:34.371412 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.371392 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw5wp\" (UniqueName: \"kubernetes.io/projected/6a319908-03ee-432c-ab8c-6f76a64df422-kube-api-access-mw5wp\") pod \"network-check-target-lhk4f\" (UID: \"6a319908-03ee-432c-ab8c-6f76a64df422\") " pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:34.428932 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.428839 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4mdrn\"" Jun 11 22:53:34.431984 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.431968 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbjj6" Jun 11 22:53:34.436825 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.436811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:34.559504 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.559468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbjj6"] Jun 11 22:53:34.562860 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:53:34.562818 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d46664d_dfda_4138_aa94_ff3bb5cf5b8f.slice/crio-b9fa94786f48f1121e8736bb462fd06142e994dac41cc1b2cf182684fa30fa71 WatchSource:0}: Error finding container b9fa94786f48f1121e8736bb462fd06142e994dac41cc1b2cf182684fa30fa71: Status 404 returned error can't find the container with id b9fa94786f48f1121e8736bb462fd06142e994dac41cc1b2cf182684fa30fa71 Jun 11 22:53:34.576367 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.576346 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lhk4f"] Jun 11 22:53:34.579467 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:53:34.579443 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a319908_03ee_432c_ab8c_6f76a64df422.slice/crio-b075ba790806d7cc5f7b104cb9b208bd2328ca3aa387ffb053efb31863a5a8f0 WatchSource:0}: Error finding container b075ba790806d7cc5f7b104cb9b208bd2328ca3aa387ffb053efb31863a5a8f0: Status 404 returned error can't find the container with id b075ba790806d7cc5f7b104cb9b208bd2328ca3aa387ffb053efb31863a5a8f0 Jun 11 22:53:34.676795 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.676757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lhk4f" event={"ID":"6a319908-03ee-432c-ab8c-6f76a64df422","Type":"ContainerStarted","Data":"b075ba790806d7cc5f7b104cb9b208bd2328ca3aa387ffb053efb31863a5a8f0"} Jun 11 22:53:34.677913 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:34.677890 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbjj6" event={"ID":"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f","Type":"ContainerStarted","Data":"b9fa94786f48f1121e8736bb462fd06142e994dac41cc1b2cf182684fa30fa71"} Jun 11 22:53:35.356329 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:35.356288 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:53:35.356549 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:35.356460 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:53:35.356549 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:53:35.356538 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:54:07.356519874 +0000 UTC m=+99.301768746 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:53:36.043734 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:36.043695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:36.098374 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:36.098338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:36.145887 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:36.145835 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:53:40.698757 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:40.698711 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbjj6" event={"ID":"3d46664d-dfda-4138-aa94-ff3bb5cf5b8f","Type":"ContainerStarted","Data":"3cdd4ef1be63e58cce327bf9a8ec187ed5a0ba7edf1b8bcde86c979e9cbeb13c"} Jun 11 22:53:40.701234 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:40.701203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lhk4f" event={"ID":"6a319908-03ee-432c-ab8c-6f76a64df422","Type":"ContainerStarted","Data":"67ac48b5a46b199543576198c07f4ea3f4afd252064e9451930a84c127076269"} Jun 11 22:53:40.701375 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:40.701293 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:53:40.713650 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:40.713588 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cbjj6" podStartSLOduration=67.377615385 podStartE2EDuration="1m12.71357114s" totalImagesPullingTime="5.335955755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:53:40.713451952 +0000 UTC m=+72.658700868" watchObservedRunningTime="2026-06-11 22:53:40.71357114 +0000 UTC m=+72.658820031" Jun 11 22:53:40.726680 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:53:40.726640 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lhk4f" podStartSLOduration=67.40719133 podStartE2EDuration="1m12.726625688s" totalImagesPullingTime="5.319434358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:53:40.726469932 +0000 UTC m=+72.671718823" watchObservedRunningTime="2026-06-11 22:53:40.726625688 +0000 UTC m=+72.671874579" Jun 11 22:54:07.377052 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:54:07.376934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:54:07.377525 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:54:07.377071 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:54:07.377525 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:54:07.377138 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:55:11.377121404 +0000 UTC m=+163.322370293 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:54:11.707025 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:54:11.706990 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lhk4f" Jun 11 22:54:38.188329 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:54:38.188287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:54:38.188806 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:54:38.188450 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 22:54:38.188806 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:54:38.188522 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs podName:db6bbb27-ddf3-492d-96c9-9681525ea2ae nodeName:}" failed. No retries permitted until 2026-06-11 22:56:40.188505186 +0000 UTC m=+252.133754055 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs") pod "network-metrics-daemon-kq7cx" (UID: "db6bbb27-ddf3-492d-96c9-9681525ea2ae") : secret "metrics-daemon-secret" not found Jun 11 22:55:06.459767 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:06.459719 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q65s9" podUID="ba1b3401-8894-4b43-8ab1-58fe4d6ee017" Jun 11 22:55:06.944721 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:06.944693 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q65s9" Jun 11 22:55:07.419525 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:07.419439 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kq7cx" podUID="db6bbb27-ddf3-492d-96c9-9681525ea2ae" Jun 11 22:55:11.417423 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:11.417385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:55:11.417886 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:11.417499 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 22:55:11.417886 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:11.417564 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls podName:ba1b3401-8894-4b43-8ab1-58fe4d6ee017 nodeName:}" failed. No retries permitted until 2026-06-11 22:57:13.417550303 +0000 UTC m=+285.362799173 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls") pod "dns-default-q65s9" (UID: "ba1b3401-8894-4b43-8ab1-58fe4d6ee017") : secret "dns-default-metrics-tls" not found Jun 11 22:55:19.483176 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:19.483089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:22.408174 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:22.408140 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:55:33.017748 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:33.017712 2562 generic.go:356] "Generic (PLEG): container finished" podID="502f73b6-cc70-49cf-a6de-f94fcb6d084f" containerID="ea7622e9857f216871078639f190e30df6a5e425ae683df4953b4fb8e8f904e0" exitCode=1 Jun 11 22:55:33.018177 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:33.017760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" event={"ID":"502f73b6-cc70-49cf-a6de-f94fcb6d084f","Type":"ContainerDied","Data":"ea7622e9857f216871078639f190e30df6a5e425ae683df4953b4fb8e8f904e0"} Jun 11 22:55:33.018177 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:33.018104 2562 scope.go:117] "RemoveContainer" containerID="ea7622e9857f216871078639f190e30df6a5e425ae683df4953b4fb8e8f904e0" Jun 11 22:55:33.674416 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:33.674386 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:55:34.022611 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:34.022573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" event={"ID":"502f73b6-cc70-49cf-a6de-f94fcb6d084f","Type":"ContainerStarted","Data":"8a74ccc8c57d926ca560f9b6a68e8d6648b96997288f36bf9cd270ba46c11d2a"} Jun 11 22:55:34.023026 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:34.022803 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:55:34.023358 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:34.023343 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6694ddcb6b-6njwb" Jun 11 22:55:42.744076 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:42.744035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:42.839186 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:42.839159 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:45.321667 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:45.321631 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:45.422456 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:45.422414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:46.130949 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.130912 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gnbfj"] Jun 11 22:55:46.133895 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.133872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.148823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.148796 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 22:55:46.148823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.148813 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 22:55:46.149031 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.148878 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 22:55:46.149031 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.148892 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 22:55:46.149031 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.148934 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kfbj2\"" Jun 11 22:55:46.151150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.151129 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gnbfj"] Jun 11 22:55:46.157245 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.157225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:46.221196 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.221167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:46.260454 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.260425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f5fee67-6a24-476f-a02c-16d9082640ff-data-volume\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.260539 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.260463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6s5z\" (UniqueName: \"kubernetes.io/projected/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-api-access-z6s5z\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.260539 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.260524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f5fee67-6a24-476f-a02c-16d9082640ff-crio-socket\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.260611 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.260559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.260611 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.260593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361118 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361118 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f5fee67-6a24-476f-a02c-16d9082640ff-data-volume\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6s5z\" (UniqueName: \"kubernetes.io/projected/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-api-access-z6s5z\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f5fee67-6a24-476f-a02c-16d9082640ff-crio-socket\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.361227 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361289 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f5fee67-6a24-476f-a02c-16d9082640ff-crio-socket\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.361293 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:46.861276272 +0000 UTC m=+198.806525140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : secret "insights-runtime-extractor-tls" not found Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.361323 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.361412 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:46.861392065 +0000 UTC m=+198.806640951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : configmap "kube-rbac-proxy" not found Jun 11 22:55:46.361581 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.361551 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f5fee67-6a24-476f-a02c-16d9082640ff-data-volume\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.369805 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.369778 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6s5z\" (UniqueName: \"kubernetes.io/projected/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-api-access-z6s5z\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.805089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.805055 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:46.853200 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.853171 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:46.865742 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.865722 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.865803 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:46.865754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:46.865890 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.865878 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 22:55:46.865927 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.865884 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 22:55:46.865962 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.865943 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:47.865926723 +0000 UTC m=+199.811175592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : configmap "kube-rbac-proxy" not found Jun 11 22:55:46.866007 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:46.865961 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:47.865952504 +0000 UTC m=+199.811201386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : secret "insights-runtime-extractor-tls" not found Jun 11 22:55:47.243260 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.243224 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:47.303636 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.303593 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:47.581130 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.581049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:47.670389 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.670344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:47.781669 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.781630 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:47.872301 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.872218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:47.872301 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.872260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:47.872476 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:47.872328 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 22:55:47.872476 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:47.872363 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 22:55:47.872476 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:47.872404 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:49.872388268 +0000 UTC m=+201.817637159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : configmap "kube-rbac-proxy" not found Jun 11 22:55:47.872476 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:47.872422 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:49.872409962 +0000 UTC m=+201.817658831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : secret "insights-runtime-extractor-tls" not found Jun 11 22:55:47.882499 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:47.882470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:48.296818 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:48.296793 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nmkb6_149874b1-5504-4544-a52f-c69bf66ee3b3/dns-node-resolver/0.log" Jun 11 22:55:48.800267 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:48.800239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:48.900584 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:48.900550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:49.163376 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:49.163297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:49.295995 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:49.295969 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p2zlk_2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751/node-ca/0.log" Jun 11 22:55:49.696852 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:49.696809 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fcl8g_4643df29-4bfb-4cf1-ba2b-226430b8f802/serve-healthcheck-canary/0.log" Jun 11 22:55:49.889879 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:49.889815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:49.889879 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:49.889883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:49.890291 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:49.889968 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 22:55:49.890291 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:49.890044 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:53.890027994 +0000 UTC m=+205.835276878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : secret "insights-runtime-extractor-tls" not found Jun 11 22:55:49.890291 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:49.889978 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 22:55:49.890291 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:49.890109 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:55:53.890095471 +0000 UTC m=+205.835344344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : configmap "kube-rbac-proxy" not found Jun 11 22:55:49.991179 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:49.991095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:50.804481 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:50.804451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:51.814498 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:51.814464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:52.256499 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:52.256463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:52.493711 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:52.493672 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:52.697276 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:52.697241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:53.105502 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:53.105413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:53.305045 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:53.305008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:53.738286 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:53.738243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:53.919148 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:53.919112 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:53.919345 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:53.919211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:55:53.919345 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:53.919232 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 22:55:53.919345 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:53.919301 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:56:01.91928733 +0000 UTC m=+213.864536199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : configmap "kube-rbac-proxy" not found Jun 11 22:55:53.919345 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:53.919320 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 22:55:53.919513 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:55:53.919373 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls podName:2f5fee67-6a24-476f-a02c-16d9082640ff nodeName:}" failed. No retries permitted until 2026-06-11 22:56:01.919357563 +0000 UTC m=+213.864606434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gnbfj" (UID: "2f5fee67-6a24-476f-a02c-16d9082640ff") : secret "insights-runtime-extractor-tls" not found Jun 11 22:55:54.299106 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:54.299069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:54.690445 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:54.690408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:54.790734 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:54.790707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:54.892954 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:54.892927 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:55.101525 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:55.101440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:55.492319 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:55.492280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:55.734448 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:55.734411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:55.894541 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:55.894459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:57.104503 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:57.104468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:57.308717 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:57.308688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:57.408315 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:57.408231 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:57.510314 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:57.510283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:57.695604 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:57.695570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:57.900872 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:57.900827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:58.891374 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:58.891344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:59.250760 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:59.250729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:59.345314 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:59.345278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:59.497343 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:59.497310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:55:59.689838 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:55:59.689804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:00.095778 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:00.095672 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:00.546020 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:00.545985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:00.650875 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:00.650818 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:00.745568 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:00.745539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:00.900079 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:00.899986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:01.068096 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.068063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:01.169857 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.169749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:01.299663 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.299631 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:01.499229 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.499191 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:01.894159 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.894078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:01.978665 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.978632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:56:01.978837 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.978671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:56:01.979267 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.979248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5fee67-6a24-476f-a02c-16d9082640ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:56:01.980892 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:01.980872 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5fee67-6a24-476f-a02c-16d9082640ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gnbfj\" (UID: \"2f5fee67-6a24-476f-a02c-16d9082640ff\") " pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:56:02.042161 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:02.042126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gnbfj" Jun 11 22:56:02.175632 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:02.175555 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gnbfj"] Jun 11 22:56:02.178425 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:02.178393 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5fee67_6a24_476f_a02c_16d9082640ff.slice/crio-194c8cb5aaf9883da7ac2b1f41df5087c5fac5a196a33f50a529387e6cf6a447 WatchSource:0}: Error finding container 194c8cb5aaf9883da7ac2b1f41df5087c5fac5a196a33f50a529387e6cf6a447: Status 404 returned error can't find the container with id 194c8cb5aaf9883da7ac2b1f41df5087c5fac5a196a33f50a529387e6cf6a447 Jun 11 22:56:03.108683 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:03.108647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gnbfj" event={"ID":"2f5fee67-6a24-476f-a02c-16d9082640ff","Type":"ContainerStarted","Data":"194c8cb5aaf9883da7ac2b1f41df5087c5fac5a196a33f50a529387e6cf6a447"} Jun 11 22:56:06.119262 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:06.119179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gnbfj" event={"ID":"2f5fee67-6a24-476f-a02c-16d9082640ff","Type":"ContainerStarted","Data":"aaebf471f1d7e0b30322c5c42a59eb367526a070f0ac5166dd98f4f917496188"} Jun 11 22:56:07.124103 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:07.124065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gnbfj" event={"ID":"2f5fee67-6a24-476f-a02c-16d9082640ff","Type":"ContainerStarted","Data":"a4c5fdf2adedba80cca49e7a11b1299d99ce4d9b163de64b4f16453d0c6e3792"} Jun 11 22:56:08.128551 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:08.128519 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gnbfj" event={"ID":"2f5fee67-6a24-476f-a02c-16d9082640ff","Type":"ContainerStarted","Data":"e821d2b4ad56232730621bc3181cef4e2943b472e2fd11ed160b2941658aee5b"} Jun 11 22:56:08.146559 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:08.146518 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gnbfj" podStartSLOduration=16.413871257 podStartE2EDuration="22.146505064s" totalImagesPullingTime="5.732633807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 22:55:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:08.145162149 +0000 UTC m=+220.090411041" watchObservedRunningTime="2026-06-11 22:56:08.146505064 +0000 UTC m=+220.091753953" Jun 11 22:56:08.988918 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:08.988879 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr"] Jun 11 22:56:08.992022 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:08.992008 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:08.995839 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:08.995823 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ln996\"" Jun 11 22:56:08.995943 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:08.995828 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 11 22:56:09.002266 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.002242 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr"] Jun 11 22:56:09.069666 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.069635 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.133960 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.133932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0d04c6c0-1605-4ac2-a33f-56e93e47abc7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-xccvr\" (UID: \"0d04c6c0-1605-4ac2-a33f-56e93e47abc7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:09.159989 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.159958 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.235179 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.235150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0d04c6c0-1605-4ac2-a33f-56e93e47abc7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-xccvr\" (UID: \"0d04c6c0-1605-4ac2-a33f-56e93e47abc7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:09.237721 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.237688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0d04c6c0-1605-4ac2-a33f-56e93e47abc7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-xccvr\" (UID: \"0d04c6c0-1605-4ac2-a33f-56e93e47abc7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:09.260984 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.260909 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.300089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.300056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:09.399393 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.399368 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-694c955b9-hhs62"] Jun 11 22:56:09.403956 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.403934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:09.407756 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.407731 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 22:56:09.407915 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.407901 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-czgmk\"" Jun 11 22:56:09.408191 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.408161 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 22:56:09.417493 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.417452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.418650 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.418630 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-hhs62"] Jun 11 22:56:09.432753 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.432731 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr"] Jun 11 22:56:09.435264 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:09.435244 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d04c6c0_1605_4ac2_a33f_56e93e47abc7.slice/crio-7a923776edd57baf09c23c47e2c5bb511763a0578fa19189086ead5b4d164321 WatchSource:0}: Error finding container 7a923776edd57baf09c23c47e2c5bb511763a0578fa19189086ead5b4d164321: Status 404 returned error can't find the container with id 7a923776edd57baf09c23c47e2c5bb511763a0578fa19189086ead5b4d164321 Jun 11 22:56:09.486526 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.486493 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.537195 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.537122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j5hv\" (UniqueName: \"kubernetes.io/projected/1f9a17f6-e92f-4701-ad00-2b4e1216e004-kube-api-access-4j5hv\") pod \"downloads-694c955b9-hhs62\" (UID: \"1f9a17f6-e92f-4701-ad00-2b4e1216e004\") " pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:09.539739 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.539710 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.638085 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.638056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j5hv\" (UniqueName: \"kubernetes.io/projected/1f9a17f6-e92f-4701-ad00-2b4e1216e004-kube-api-access-4j5hv\") pod \"downloads-694c955b9-hhs62\" (UID: \"1f9a17f6-e92f-4701-ad00-2b4e1216e004\") " pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:09.646188 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.646158 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j5hv\" (UniqueName: \"kubernetes.io/projected/1f9a17f6-e92f-4701-ad00-2b4e1216e004-kube-api-access-4j5hv\") pod \"downloads-694c955b9-hhs62\" (UID: \"1f9a17f6-e92f-4701-ad00-2b4e1216e004\") " pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:09.683048 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.683006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:09.712917 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.712890 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:09.833011 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:09.832935 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-hhs62"] Jun 11 22:56:09.835671 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:09.835645 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9a17f6_e92f_4701_ad00_2b4e1216e004.slice/crio-8eaa5fe40a926d656c59d8702bc5cac6ebfbe4b5260a86902e6edc6c75f61584 WatchSource:0}: Error finding container 8eaa5fe40a926d656c59d8702bc5cac6ebfbe4b5260a86902e6edc6c75f61584: Status 404 returned error can't find the container with id 8eaa5fe40a926d656c59d8702bc5cac6ebfbe4b5260a86902e6edc6c75f61584 Jun 11 22:56:10.058827 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.058789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:10.112507 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.112418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:10.136925 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.136885 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-hhs62" event={"ID":"1f9a17f6-e92f-4701-ad00-2b4e1216e004","Type":"ContainerStarted","Data":"8eaa5fe40a926d656c59d8702bc5cac6ebfbe4b5260a86902e6edc6c75f61584"} Jun 11 22:56:10.138254 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.138219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" event={"ID":"0d04c6c0-1605-4ac2-a33f-56e93e47abc7","Type":"ContainerStarted","Data":"7a923776edd57baf09c23c47e2c5bb511763a0578fa19189086ead5b4d164321"} Jun 11 22:56:10.469692 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.469664 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:10.576797 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.576760 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:10.843679 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.843626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:10.902721 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:10.902670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:11.143192 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.143103 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" event={"ID":"0d04c6c0-1605-4ac2-a33f-56e93e47abc7","Type":"ContainerStarted","Data":"57b3593698836faec0db51b66acc51f53b8ba76dbd2d70724bdfa4af1a44cbf4"} Jun 11 22:56:11.143583 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.143428 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:11.148882 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.148860 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" Jun 11 22:56:11.160359 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.160317 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-xccvr" podStartSLOduration=2.211544404 podStartE2EDuration="3.160305268s" totalImagesPullingTime="948.760864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:11.159581306 +0000 UTC m=+223.104830218" watchObservedRunningTime="2026-06-11 22:56:11.160305268 +0000 UTC m=+223.105554158" Jun 11 22:56:11.282006 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.281966 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:11.469057 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.469020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:11.716717 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.716651 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:11.781515 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:11.781440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:12.052984 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:12.052904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:12.496432 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:12.496391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:12.558898 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:12.558859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:14.422295 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:14.422243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:15.851146 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:15.851111 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.065643 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.065606 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.126002 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.125925 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pqvxx"] Jun 11 22:56:16.129388 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.129367 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.132310 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.132290 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 22:56:16.132438 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.132362 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 22:56:16.132506 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.132442 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 22:56:16.132569 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.132511 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 22:56:16.132622 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.132593 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9zjvk\"" Jun 11 22:56:16.132666 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.132631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 22:56:16.133797 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.133772 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 22:56:16.165157 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.165101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.258588 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.258550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.290653 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-wtmp\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.290823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-tls\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.290823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-root\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.290823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-textfile\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.290989 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.290989 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef85d25-218d-4b67-b9c7-35aedadf481c-metrics-client-ca\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.290989 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-accelerators-collector-config\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.291136 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.290993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-sys\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.291136 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.291012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brng8\" (UniqueName: \"kubernetes.io/projected/3ef85d25-218d-4b67-b9c7-35aedadf481c-kube-api-access-brng8\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392257 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392163 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-tls\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392431 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-root\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392431 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-textfile\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392431 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:56:16.392345 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 22:56:16.392597 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392434 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-root\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392597 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:56:16.392498 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-tls podName:3ef85d25-218d-4b67-b9c7-35aedadf481c nodeName:}" failed. No retries permitted until 2026-06-11 22:56:16.892474347 +0000 UTC m=+228.837723238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-tls") pod "node-exporter-pqvxx" (UID: "3ef85d25-218d-4b67-b9c7-35aedadf481c") : secret "node-exporter-tls" not found Jun 11 22:56:16.392597 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392597 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef85d25-218d-4b67-b9c7-35aedadf481c-metrics-client-ca\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-accelerators-collector-config\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-sys\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brng8\" (UniqueName: \"kubernetes.io/projected/3ef85d25-218d-4b67-b9c7-35aedadf481c-kube-api-access-brng8\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-wtmp\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.392823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392796 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-textfile\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.393235 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392913 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-wtmp\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.393235 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.392918 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ef85d25-218d-4b67-b9c7-35aedadf481c-sys\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.393343 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.393255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef85d25-218d-4b67-b9c7-35aedadf481c-metrics-client-ca\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.393416 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.393394 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-accelerators-collector-config\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.394961 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.394940 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.409167 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.409148 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brng8\" (UniqueName: \"kubernetes.io/projected/3ef85d25-218d-4b67-b9c7-35aedadf481c-kube-api-access-brng8\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.430081 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.430049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.543284 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.543252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.638372 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.638335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.736904 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.736868 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:16.898538 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.898499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-tls\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:16.901209 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:16.901187 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ef85d25-218d-4b67-b9c7-35aedadf481c-node-exporter-tls\") pod \"node-exporter-pqvxx\" (UID: \"3ef85d25-218d-4b67-b9c7-35aedadf481c\") " pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:17.041116 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:17.041040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pqvxx" Jun 11 22:56:17.052758 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:17.052717 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef85d25_218d_4b67_b9c7_35aedadf481c.slice/crio-474e77d04f8a1f7d00954415a1333f81830996ed8784b95bef27db8d823d084a WatchSource:0}: Error finding container 474e77d04f8a1f7d00954415a1333f81830996ed8784b95bef27db8d823d084a: Status 404 returned error can't find the container with id 474e77d04f8a1f7d00954415a1333f81830996ed8784b95bef27db8d823d084a Jun 11 22:56:17.108515 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:17.108463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:17.166302 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:17.166268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqvxx" event={"ID":"3ef85d25-218d-4b67-b9c7-35aedadf481c","Type":"ContainerStarted","Data":"474e77d04f8a1f7d00954415a1333f81830996ed8784b95bef27db8d823d084a"} Jun 11 22:56:17.180135 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:17.180021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:17.350475 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:17.350387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:18.171955 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:18.171918 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqvxx" event={"ID":"3ef85d25-218d-4b67-b9c7-35aedadf481c","Type":"ContainerStarted","Data":"1adcc1a184a2bffdc7c1a1188f4899b17521264d16ec8b6f9be919afee056208"} Jun 11 22:56:19.010823 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.010789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:19.105517 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.105482 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6b45c4cffb-7972f"] Jun 11 22:56:19.106116 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.106040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:19.109429 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.109409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.112115 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.112085 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 11 22:56:19.112115 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.112097 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 11 22:56:19.112375 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.112097 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 11 22:56:19.112375 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.112363 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 11 22:56:19.112488 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.112444 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-57js6\"" Jun 11 22:56:19.112605 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.112588 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 11 22:56:19.113198 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.113179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-68h5p5p9jf7kl\"" Jun 11 22:56:19.119432 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.119092 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b45c4cffb-7972f"] Jun 11 22:56:19.159861 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.159814 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:19.177375 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.177346 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ef85d25-218d-4b67-b9c7-35aedadf481c" containerID="1adcc1a184a2bffdc7c1a1188f4899b17521264d16ec8b6f9be919afee056208" exitCode=0 Jun 11 22:56:19.177780 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.177380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqvxx" event={"ID":"3ef85d25-218d-4b67-b9c7-35aedadf481c","Type":"ContainerDied","Data":"1adcc1a184a2bffdc7c1a1188f4899b17521264d16ec8b6f9be919afee056208"} Jun 11 22:56:19.220386 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220361 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2k66\" (UniqueName: \"kubernetes.io/projected/17f0432b-f005-4c55-b1a4-210bba424093-kube-api-access-h2k66\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220543 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220543 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220543 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220461 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17f0432b-f005-4c55-b1a4-210bba424093-metrics-client-ca\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-tls\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.220681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.220650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-grpc-tls\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.298490 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.298402 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:19.301978 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.301956 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84b598db68-5hrcp"] Jun 11 22:56:19.304926 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.304908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.307720 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.307700 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 22:56:19.307830 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.307761 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 22:56:19.307918 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.307835 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lr9tq\"" Jun 11 22:56:19.307994 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.307977 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 22:56:19.308192 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.308168 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 22:56:19.308485 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.308464 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 22:56:19.316612 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.316589 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b598db68-5hrcp"] Jun 11 22:56:19.321877 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.321826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.321985 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.321900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.321985 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.321930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-tls\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.322102 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.322032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-grpc-tls\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.322102 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.322081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2k66\" (UniqueName: \"kubernetes.io/projected/17f0432b-f005-4c55-b1a4-210bba424093-kube-api-access-h2k66\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.322195 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.322132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.322307 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.322256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.322372 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.322361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17f0432b-f005-4c55-b1a4-210bba424093-metrics-client-ca\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.323124 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.323097 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17f0432b-f005-4c55-b1a4-210bba424093-metrics-client-ca\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.324944 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.324893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.324944 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.324936 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.325196 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.325172 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.325473 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.325453 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-tls\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.325530 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.325471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-grpc-tls\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.325658 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.325637 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17f0432b-f005-4c55-b1a4-210bba424093-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.332526 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.332504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2k66\" (UniqueName: \"kubernetes.io/projected/17f0432b-f005-4c55-b1a4-210bba424093-kube-api-access-h2k66\") pod \"thanos-querier-6b45c4cffb-7972f\" (UID: \"17f0432b-f005-4c55-b1a4-210bba424093\") " pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.421755 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.421708 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:19.422938 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.422904 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-console-config\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.422938 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.422941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-service-ca\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.423124 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.422972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-oauth-config\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.423124 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.423002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-serving-cert\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.423124 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.423035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-oauth-serving-cert\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.423124 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.423094 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbxbh\" (UniqueName: \"kubernetes.io/projected/a5459afb-0abf-4f18-a271-b6ce127096b6-kube-api-access-vbxbh\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.523572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.523532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-console-config\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.523572 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.523583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-service-ca\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.523812 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.523709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-oauth-config\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.523812 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.523755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-serving-cert\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.523812 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.523796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-oauth-serving-cert\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.524021 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.523894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbxbh\" (UniqueName: \"kubernetes.io/projected/a5459afb-0abf-4f18-a271-b6ce127096b6-kube-api-access-vbxbh\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.524412 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.524361 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-service-ca\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.524731 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.524704 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-oauth-serving-cert\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.524731 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.524717 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-console-config\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.526510 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.526480 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-serving-cert\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.526608 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.526521 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-oauth-config\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.531620 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.531551 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbxbh\" (UniqueName: \"kubernetes.io/projected/a5459afb-0abf-4f18-a271-b6ce127096b6-kube-api-access-vbxbh\") pod \"console-84b598db68-5hrcp\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:19.616904 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:19.616809 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:20.549577 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.549547 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78d6c5fdf-hqdfq"] Jun 11 22:56:20.552588 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.552569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.555608 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.555583 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 11 22:56:20.555721 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.555657 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pfhbh\"" Jun 11 22:56:20.556968 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.556946 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4p903hnsnc2fh\"" Jun 11 22:56:20.556968 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.556968 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 22:56:20.557148 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.556974 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 11 22:56:20.557148 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.557025 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 11 22:56:20.566824 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.566802 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78d6c5fdf-hqdfq"] Jun 11 22:56:20.735973 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.735930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-secret-metrics-server-client-certs\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.736147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.735991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-client-ca-bundle\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.736147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.736026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-audit-log\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.736147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.736067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-metrics-server-audit-profiles\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.736147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.736097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmd4t\" (UniqueName: \"kubernetes.io/projected/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-kube-api-access-lmd4t\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.736358 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.736150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-secret-metrics-server-tls\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.736358 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.736207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837070 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.836964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-audit-log\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837070 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-metrics-server-audit-profiles\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837070 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmd4t\" (UniqueName: \"kubernetes.io/projected/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-kube-api-access-lmd4t\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837347 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-secret-metrics-server-tls\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837347 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837347 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-secret-metrics-server-client-certs\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837347 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-client-ca-bundle\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.837565 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.837410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-audit-log\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.838463 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.838430 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.838590 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.838536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-metrics-server-audit-profiles\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.840250 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.840189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-client-ca-bundle\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.840748 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.840728 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-secret-metrics-server-tls\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.840894 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.840871 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-secret-metrics-server-client-certs\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.844677 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.844654 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmd4t\" (UniqueName: \"kubernetes.io/projected/49314bf2-cb4f-4880-aa1f-f05e709bc0d4-kube-api-access-lmd4t\") pod \"metrics-server-78d6c5fdf-hqdfq\" (UID: \"49314bf2-cb4f-4880-aa1f-f05e709bc0d4\") " pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.864443 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.864383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:20.874707 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.874673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:20.875250 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.875230 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-67jhk"] Jun 11 22:56:20.878163 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.878147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:20.880628 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.880606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 11 22:56:20.880785 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.880769 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n5vwr\"" Jun 11 22:56:20.886141 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.885653 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-67jhk"] Jun 11 22:56:20.931904 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:20.931873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:21.039115 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.039080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.039265 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.039203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v68b9\" (UniqueName: \"kubernetes.io/projected/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-kube-api-access-v68b9\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.140559 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.140476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.140712 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.140587 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v68b9\" (UniqueName: \"kubernetes.io/projected/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-kube-api-access-v68b9\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.140712 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:56:21.140643 2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 11 22:56:21.140805 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:56:21.140723 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-monitoring-plugin-cert podName:a81f0829-d23d-4cdf-b4f0-3d71d925e7c3 nodeName:}" failed. No retries permitted until 2026-06-11 22:56:21.640700772 +0000 UTC m=+233.585949646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-monitoring-plugin-cert") pod "monitoring-plugin-589576dfc7-67jhk" (UID: "a81f0829-d23d-4cdf-b4f0-3d71d925e7c3") : secret "monitoring-plugin-cert" not found Jun 11 22:56:21.151292 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.151259 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v68b9\" (UniqueName: \"kubernetes.io/projected/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-kube-api-access-v68b9\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.349053 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.349008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:21.646094 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.646039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.648726 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.648700 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a81f0829-d23d-4cdf-b4f0-3d71d925e7c3-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-67jhk\" (UID: \"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:21.789413 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:21.789371 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:22.299253 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:22.299217 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:23.299336 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:23.299303 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:23.364525 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:23.364491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:24.702553 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:24.702520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:24.799346 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:24.799310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:25.618126 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:25.617968 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b45c4cffb-7972f"] Jun 11 22:56:25.619355 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:25.619325 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f0432b_f005_4c55_b1a4_210bba424093.slice/crio-348cad794e4f7c8cc5d6dd81fc2f2a6875a5b86a2020937adc25388e9b24a769 WatchSource:0}: Error finding container 348cad794e4f7c8cc5d6dd81fc2f2a6875a5b86a2020937adc25388e9b24a769: Status 404 returned error can't find the container with id 348cad794e4f7c8cc5d6dd81fc2f2a6875a5b86a2020937adc25388e9b24a769 Jun 11 22:56:25.840643 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:25.840615 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b598db68-5hrcp"] Jun 11 22:56:25.843094 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:25.843062 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5459afb_0abf_4f18_a271_b6ce127096b6.slice/crio-5256b3d0351e9d07575ac96df066968db0f2362ee794bd4df9b0360db67dc5d8 WatchSource:0}: Error finding container 5256b3d0351e9d07575ac96df066968db0f2362ee794bd4df9b0360db67dc5d8: Status 404 returned error can't find the container with id 5256b3d0351e9d07575ac96df066968db0f2362ee794bd4df9b0360db67dc5d8 Jun 11 22:56:25.846253 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:25.846182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-67jhk"] Jun 11 22:56:25.847431 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:25.847399 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78d6c5fdf-hqdfq"] Jun 11 22:56:25.849011 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:25.848973 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49314bf2_cb4f_4880_aa1f_f05e709bc0d4.slice/crio-898435e37a565f839a02a6ebf21ea8d42bdfb07992d2a7370cc840252dde8af7 WatchSource:0}: Error finding container 898435e37a565f839a02a6ebf21ea8d42bdfb07992d2a7370cc840252dde8af7: Status 404 returned error can't find the container with id 898435e37a565f839a02a6ebf21ea8d42bdfb07992d2a7370cc840252dde8af7 Jun 11 22:56:25.849939 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:25.849919 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81f0829_d23d_4cdf_b4f0_3d71d925e7c3.slice/crio-3490d337e8d72bf3bba051dd3bccdccd8a210241cf337ef143b6ce1eca9e2574 WatchSource:0}: Error finding container 3490d337e8d72bf3bba051dd3bccdccd8a210241cf337ef143b6ce1eca9e2574: Status 404 returned error can't find the container with id 3490d337e8d72bf3bba051dd3bccdccd8a210241cf337ef143b6ce1eca9e2574 Jun 11 22:56:26.203871 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.203506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-hhs62" event={"ID":"1f9a17f6-e92f-4701-ad00-2b4e1216e004","Type":"ContainerStarted","Data":"9c6a2b81f758bc171ec333824ccaebc66fbababe998002a95f40043627770ed7"} Jun 11 22:56:26.204454 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.204377 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:26.208717 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.207982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqvxx" event={"ID":"3ef85d25-218d-4b67-b9c7-35aedadf481c","Type":"ContainerStarted","Data":"6df6e3ee395b392cb0cfcb927548f710360530b8c17d7c9d4193102be9c77483"} Jun 11 22:56:26.208717 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.208041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pqvxx" event={"ID":"3ef85d25-218d-4b67-b9c7-35aedadf481c","Type":"ContainerStarted","Data":"dada98d3c71ec2569d5d37b8fc772018c488c8b5c581a26111906ee6a2007f91"} Jun 11 22:56:26.209545 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.209516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" event={"ID":"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3","Type":"ContainerStarted","Data":"3490d337e8d72bf3bba051dd3bccdccd8a210241cf337ef143b6ce1eca9e2574"} Jun 11 22:56:26.211002 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.210980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b598db68-5hrcp" event={"ID":"a5459afb-0abf-4f18-a271-b6ce127096b6","Type":"ContainerStarted","Data":"5256b3d0351e9d07575ac96df066968db0f2362ee794bd4df9b0360db67dc5d8"} Jun 11 22:56:26.212373 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.212349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" event={"ID":"49314bf2-cb4f-4880-aa1f-f05e709bc0d4","Type":"ContainerStarted","Data":"898435e37a565f839a02a6ebf21ea8d42bdfb07992d2a7370cc840252dde8af7"} Jun 11 22:56:26.214411 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.214383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"348cad794e4f7c8cc5d6dd81fc2f2a6875a5b86a2020937adc25388e9b24a769"} Jun 11 22:56:26.215703 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.215684 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-694c955b9-hhs62" Jun 11 22:56:26.227681 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.227531 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-694c955b9-hhs62" podStartSLOduration=1.506146485 podStartE2EDuration="17.227513958s" totalImagesPullingTime="15.721367473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:26.226383395 +0000 UTC m=+238.171632309" watchObservedRunningTime="2026-06-11 22:56:26.227513958 +0000 UTC m=+238.172762850" Jun 11 22:56:26.271358 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.270934 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pqvxx" podStartSLOduration=9.20691104 podStartE2EDuration="10.270912378s" totalImagesPullingTime="991.644169ms" totalInitContainerRuntime="72.357169ms" isStatefulPod=true podCreationTimestamp="2026-06-11 22:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:26.269734839 +0000 UTC m=+238.214983730" watchObservedRunningTime="2026-06-11 22:56:26.270912378 +0000 UTC m=+238.216161270" Jun 11 22:56:26.791138 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:26.790361 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:27.637567 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:27.637275 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" podUID="0335807d-c705-445c-bbe7-63368f2effdd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 22:56:29.088580 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:29.088544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:29.190293 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:29.190217 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:30.115033 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:30.114994 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:30.173916 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:30.173820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:32.242054 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.242021 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" event={"ID":"a81f0829-d23d-4cdf-b4f0-3d71d925e7c3","Type":"ContainerStarted","Data":"19a9471bcaaff19ad22920b5c02fa681ade0d4d3ae9e02660eca7eb3a0191107"} Jun 11 22:56:32.242589 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.242209 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:32.243818 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.243788 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b598db68-5hrcp" event={"ID":"a5459afb-0abf-4f18-a271-b6ce127096b6","Type":"ContainerStarted","Data":"626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0"} Jun 11 22:56:32.245717 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.245695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" event={"ID":"49314bf2-cb4f-4880-aa1f-f05e709bc0d4","Type":"ContainerStarted","Data":"65d55c29bb2061d5e54b423af0a113f22c7d818b32e2e64246529ecaa77361da"} Jun 11 22:56:32.247901 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.247875 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" Jun 11 22:56:32.248150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.248132 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"42d09f1db41365e347ceec8cc6d3e673adf792d87b9738d8adfc8389fafa15f9"} Jun 11 22:56:32.248249 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.248192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"f94fc68889fd9be837081c7698e06454bc74ca375ad0b9729fe1c9df651230ef"} Jun 11 22:56:32.248249 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.248239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"e46509dccb9650a7024ee4cd7a4210086e138a8228bd4f69ca8e2d5e04f58a92"} Jun 11 22:56:32.258177 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.258130 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-589576dfc7-67jhk" podStartSLOduration=6.699184045 podStartE2EDuration="12.258113305s" totalImagesPullingTime="5.55892926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:32.256469821 +0000 UTC m=+244.201718714" watchObservedRunningTime="2026-06-11 22:56:32.258113305 +0000 UTC m=+244.203362177" Jun 11 22:56:32.274790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.274738 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84b598db68-5hrcp" podStartSLOduration=7.701979018 podStartE2EDuration="13.27472495s" totalImagesPullingTime="5.572745932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:32.273306697 +0000 UTC m=+244.218555588" watchObservedRunningTime="2026-06-11 22:56:32.27472495 +0000 UTC m=+244.219973840" Jun 11 22:56:32.309318 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:32.309264 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" podStartSLOduration=6.750154478 podStartE2EDuration="12.309242792s" totalImagesPullingTime="5.559088314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:32.290928358 +0000 UTC m=+244.236177250" watchObservedRunningTime="2026-06-11 22:56:32.309242792 +0000 UTC m=+244.254491684" Jun 11 22:56:33.254341 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:33.254310 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"6ea3ec7e63b4d62c0a0442e3dc354578f157a3a9bbe483b85b9f4ba518e80fa0"} Jun 11 22:56:34.262069 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:34.262029 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"22458081f10a7e0f4e6ee5a230e557da6181a5a2aa00ed8a22861e76c67b9e09"} Jun 11 22:56:34.262497 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:34.262116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" event={"ID":"17f0432b-f005-4c55-b1a4-210bba424093","Type":"ContainerStarted","Data":"070036bcc380f7d861e01318b2009283c63431020bf71b0ff121b9ecd40663ec"} Jun 11 22:56:34.286710 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:34.286654 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" podStartSLOduration=8.255049052 podStartE2EDuration="15.28663672s" totalImagesPullingTime="7.031587668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:56:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:34.285033044 +0000 UTC m=+246.230281956" watchObservedRunningTime="2026-06-11 22:56:34.28663672 +0000 UTC m=+246.231885611" Jun 11 22:56:35.266648 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:35.266617 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:36.275561 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:36.275537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6b45c4cffb-7972f" Jun 11 22:56:37.637034 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:37.636996 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" podUID="0335807d-c705-445c-bbe7-63368f2effdd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 22:56:39.617818 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:39.617781 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:39.618288 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:39.617886 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:56:39.619262 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:39.619241 2562 patch_prober.go:28] interesting pod/console-84b598db68-5hrcp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body= Jun 11 22:56:39.619345 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:39.619285 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-84b598db68-5hrcp" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" Jun 11 22:56:40.198132 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.198095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:56:40.200339 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.200307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db6bbb27-ddf3-492d-96c9-9681525ea2ae-metrics-certs\") pod \"network-metrics-daemon-kq7cx\" (UID: \"db6bbb27-ddf3-492d-96c9-9681525ea2ae\") " pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:56:40.411318 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.411293 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s94r6\"" Jun 11 22:56:40.419477 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.419457 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq7cx" Jun 11 22:56:40.566801 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.566771 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kq7cx"] Jun 11 22:56:40.569712 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:56:40.569677 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6bbb27_ddf3_492d_96c9_9681525ea2ae.slice/crio-646e2bd73e8a6f3242ab78e36087a469732ae172a3aadfb7cabb6190e59629d2 WatchSource:0}: Error finding container 646e2bd73e8a6f3242ab78e36087a469732ae172a3aadfb7cabb6190e59629d2: Status 404 returned error can't find the container with id 646e2bd73e8a6f3242ab78e36087a469732ae172a3aadfb7cabb6190e59629d2 Jun 11 22:56:40.846386 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.846311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:40.865099 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.865071 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:40.865220 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.865128 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:56:40.946511 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:40.946482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:41.288636 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:41.288604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kq7cx" event={"ID":"db6bbb27-ddf3-492d-96c9-9681525ea2ae","Type":"ContainerStarted","Data":"646e2bd73e8a6f3242ab78e36087a469732ae172a3aadfb7cabb6190e59629d2"} Jun 11 22:56:41.901913 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:41.901878 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:42.002715 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:42.002674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:42.293615 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:42.293593 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kq7cx" event={"ID":"db6bbb27-ddf3-492d-96c9-9681525ea2ae","Type":"ContainerStarted","Data":"53a6931f0d2e9d9736249516d6b2e0ebb9ae9a4139e6c5052fe1b2338412fc49"} Jun 11 22:56:43.298471 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:43.298434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kq7cx" event={"ID":"db6bbb27-ddf3-492d-96c9-9681525ea2ae","Type":"ContainerStarted","Data":"ec4186cb2b018960cfef028cdd6470110b4abd976bec44573d06a0a2d60ca787"} Jun 11 22:56:43.315320 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:43.315270 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kq7cx" podStartSLOduration=253.842010054 podStartE2EDuration="4m15.315255556s" totalImagesPullingTime="1.473245502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:56:43.313492697 +0000 UTC m=+255.258741587" watchObservedRunningTime="2026-06-11 22:56:43.315255556 +0000 UTC m=+255.260504445" Jun 11 22:56:43.413716 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:43.413685 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:43.483277 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:43.483244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:47.636983 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:47.636903 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" podUID="0335807d-c705-445c-bbe7-63368f2effdd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 22:56:47.636983 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:47.636969 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" Jun 11 22:56:47.637445 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:47.637409 2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"88078b0874d2d994a992c68a93ef23f7463809068e6d1c818c84d07938398c04"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 11 22:56:47.637482 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:47.637470 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" podUID="0335807d-c705-445c-bbe7-63368f2effdd" containerName="service-proxy" containerID="cri-o://88078b0874d2d994a992c68a93ef23f7463809068e6d1c818c84d07938398c04" gracePeriod=30 Jun 11 22:56:48.316665 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:48.316631 2562 generic.go:356] "Generic (PLEG): container finished" podID="0335807d-c705-445c-bbe7-63368f2effdd" containerID="88078b0874d2d994a992c68a93ef23f7463809068e6d1c818c84d07938398c04" exitCode=2 Jun 11 22:56:48.316934 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:48.316693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" event={"ID":"0335807d-c705-445c-bbe7-63368f2effdd","Type":"ContainerDied","Data":"88078b0874d2d994a992c68a93ef23f7463809068e6d1c818c84d07938398c04"} Jun 11 22:56:48.316934 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:48.316724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84cd6c48bf-28lgd" event={"ID":"0335807d-c705-445c-bbe7-63368f2effdd","Type":"ContainerStarted","Data":"cbbb97f1477fd77962c69655ce9f2f5a41a3da6195e7ccd44cf02b36566f52d5"} Jun 11 22:56:48.828086 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:48.828052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:48.929196 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:48.929163 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:49.617943 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:49.617911 2562 patch_prober.go:28] interesting pod/console-84b598db68-5hrcp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body= Jun 11 22:56:49.618129 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:49.617964 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-84b598db68-5hrcp" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" Jun 11 22:56:49.698553 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:49.698522 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:49.755538 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:49.755509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:55.619308 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:55.619273 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78d6c5fdf-hqdfq_49314bf2-cb4f-4880-aa1f-f05e709bc0d4/metrics-server/0.log" Jun 11 22:56:55.818410 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:55.818381 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-67jhk_a81f0829-d23d-4cdf-b4f0-3d71d925e7c3/monitoring-plugin/0.log" Jun 11 22:56:55.853519 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:55.853484 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:55.904260 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:55.904178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:56.617817 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:56.617792 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqvxx_3ef85d25-218d-4b67-b9c7-35aedadf481c/init-textfile/0.log" Jun 11 22:56:56.819390 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:56.819365 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqvxx_3ef85d25-218d-4b67-b9c7-35aedadf481c/node-exporter/0.log" Jun 11 22:56:57.018223 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:57.018193 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqvxx_3ef85d25-218d-4b67-b9c7-35aedadf481c/kube-rbac-proxy/0.log" Jun 11 22:56:58.409659 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:58.409624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:56:59.617492 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:59.617462 2562 patch_prober.go:28] interesting pod/console-84b598db68-5hrcp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body= Jun 11 22:56:59.617958 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:56:59.617511 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-84b598db68-5hrcp" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" probeResult="failure" output="Get \"https://10.133.0.14:8443/health\": dial tcp 10.133.0.14:8443: connect: connection refused" Jun 11 22:57:00.218801 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:00.218768 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-xccvr_0d04c6c0-1605-4ac2-a33f-56e93e47abc7/prometheus-operator-admission-webhook/0.log" Jun 11 22:57:00.419057 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:00.419027 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/thanos-query/0.log" Jun 11 22:57:00.618295 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:00.618225 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy-web/0.log" Jun 11 22:57:00.819599 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:00.819573 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy/0.log" Jun 11 22:57:00.869728 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:00.869665 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:57:00.873517 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:00.873495 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78d6c5fdf-hqdfq" Jun 11 22:57:01.018232 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:01.018203 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/prom-label-proxy/0.log" Jun 11 22:57:01.219492 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:01.219462 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy-rules/0.log" Jun 11 22:57:01.419394 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:01.419367 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy-metrics/0.log" Jun 11 22:57:02.218435 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:02.218402 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84b598db68-5hrcp_a5459afb-0abf-4f18-a271-b6ce127096b6/console/0.log" Jun 11 22:57:02.619938 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:02.619859 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-hhs62_1f9a17f6-e92f-4701-ad00-2b4e1216e004/download-server/0.log" Jun 11 22:57:03.781610 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:03.781571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:03.838656 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:03.838622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:04.418325 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:04.418300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nmkb6_149874b1-5504-4544-a52f-c69bf66ee3b3/dns-node-resolver/0.log" Jun 11 22:57:05.418140 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:05.418115 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p2zlk_2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751/node-ca/0.log" Jun 11 22:57:05.818155 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:05.818124 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fcl8g_4643df29-4bfb-4cf1-ba2b-226430b8f802/serve-healthcheck-canary/0.log" Jun 11 22:57:08.055146 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:08.055052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:08.156190 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:08.156147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:08.726214 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:08.726184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:08.827884 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:08.827826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:08.891198 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:08.891165 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84b598db68-5hrcp"] Jun 11 22:57:09.018733 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:09.018591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:09.158103 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:09.158069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:09.278588 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:09.278494 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:09.486493 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:09.486454 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:09.594204 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:09.594113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:09.945273 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:57:09.945245 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q65s9" podUID="ba1b3401-8894-4b43-8ab1-58fe4d6ee017" Jun 11 22:57:10.396820 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:10.396745 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q65s9" Jun 11 22:57:12.852442 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:12.852404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:12.953226 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:12.953195 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:13.474067 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:13.474033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:57:13.477105 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:13.477083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba1b3401-8894-4b43-8ab1-58fe4d6ee017-metrics-tls\") pod \"dns-default-q65s9\" (UID: \"ba1b3401-8894-4b43-8ab1-58fe4d6ee017\") " pod="openshift-dns/dns-default-q65s9" Jun 11 22:57:13.700285 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:13.700256 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m9z2n\"" Jun 11 22:57:13.708548 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:13.708529 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q65s9" Jun 11 22:57:13.888974 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:13.888943 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q65s9"] Jun 11 22:57:13.892030 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:57:13.892002 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1b3401_8894_4b43_8ab1_58fe4d6ee017.slice/crio-0b422eb5497e8d7921ac8b0d5d2fb87df0add55531576b1029fa1b3521f9a496 WatchSource:0}: Error finding container 0b422eb5497e8d7921ac8b0d5d2fb87df0add55531576b1029fa1b3521f9a496: Status 404 returned error can't find the container with id 0b422eb5497e8d7921ac8b0d5d2fb87df0add55531576b1029fa1b3521f9a496 Jun 11 22:57:14.414708 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:14.414677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q65s9" event={"ID":"ba1b3401-8894-4b43-8ab1-58fe4d6ee017","Type":"ContainerStarted","Data":"0b422eb5497e8d7921ac8b0d5d2fb87df0add55531576b1029fa1b3521f9a496"} Jun 11 22:57:15.847288 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:15.847257 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:15.906550 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:15.906519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:16.422509 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:16.422478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q65s9" event={"ID":"ba1b3401-8894-4b43-8ab1-58fe4d6ee017","Type":"ContainerStarted","Data":"9294121b461cf268bcf7954a53d42c10673d88fd41dad8e984783fd4585e6291"} Jun 11 22:57:17.428024 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:17.427992 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q65s9" event={"ID":"ba1b3401-8894-4b43-8ab1-58fe4d6ee017","Type":"ContainerStarted","Data":"090d31497a7e58288e1ca9025654cd0c24ca1a5503267f2cd611d77bd774ccd2"} Jun 11 22:57:17.428417 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:17.428139 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q65s9" Jun 11 22:57:17.447499 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:17.447454 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q65s9" podStartSLOduration=252.083401258 podStartE2EDuration="4m14.447436025s" totalImagesPullingTime="2.364034767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:57:17.445389827 +0000 UTC m=+289.390638719" watchObservedRunningTime="2026-06-11 22:57:17.447436025 +0000 UTC m=+289.392684917" Jun 11 22:57:25.855702 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:25.855661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:25.955141 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:25.955103 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:27.433731 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:27.433703 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q65s9" Jun 11 22:57:28.377617 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:28.377594 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:57:28.378158 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:28.378139 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 22:57:31.778860 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:31.778796 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:31.880423 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:31.880387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:31.982157 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:31.982130 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:33.914869 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:33.914798 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-84b598db68-5hrcp" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" containerID="cri-o://626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0" gracePeriod=15 Jun 11 22:57:34.152306 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.152282 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:57:34.220637 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.220601 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbxbh\" (UniqueName: \"kubernetes.io/projected/a5459afb-0abf-4f18-a271-b6ce127096b6-kube-api-access-vbxbh\") pod \"a5459afb-0abf-4f18-a271-b6ce127096b6\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " Jun 11 22:57:34.220637 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.220644 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-serving-cert\") pod \"a5459afb-0abf-4f18-a271-b6ce127096b6\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " Jun 11 22:57:34.220917 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.220687 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-service-ca\") pod \"a5459afb-0abf-4f18-a271-b6ce127096b6\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " Jun 11 22:57:34.220917 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.220712 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-oauth-config\") pod \"a5459afb-0abf-4f18-a271-b6ce127096b6\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " Jun 11 22:57:34.220917 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.220743 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-oauth-serving-cert\") pod \"a5459afb-0abf-4f18-a271-b6ce127096b6\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " Jun 11 22:57:34.220917 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.220782 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-console-config\") pod \"a5459afb-0abf-4f18-a271-b6ce127096b6\" (UID: \"a5459afb-0abf-4f18-a271-b6ce127096b6\") " Jun 11 22:57:34.221148 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.221115 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-service-ca" (OuterVolumeSpecName: "service-ca") pod "a5459afb-0abf-4f18-a271-b6ce127096b6" (UID: "a5459afb-0abf-4f18-a271-b6ce127096b6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 22:57:34.221257 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.221158 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a5459afb-0abf-4f18-a271-b6ce127096b6" (UID: "a5459afb-0abf-4f18-a271-b6ce127096b6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 22:57:34.221257 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.221206 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-console-config" (OuterVolumeSpecName: "console-config") pod "a5459afb-0abf-4f18-a271-b6ce127096b6" (UID: "a5459afb-0abf-4f18-a271-b6ce127096b6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 22:57:34.223035 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.223004 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a5459afb-0abf-4f18-a271-b6ce127096b6" (UID: "a5459afb-0abf-4f18-a271-b6ce127096b6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 22:57:34.223035 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.223014 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5459afb-0abf-4f18-a271-b6ce127096b6-kube-api-access-vbxbh" (OuterVolumeSpecName: "kube-api-access-vbxbh") pod "a5459afb-0abf-4f18-a271-b6ce127096b6" (UID: "a5459afb-0abf-4f18-a271-b6ce127096b6"). InnerVolumeSpecName "kube-api-access-vbxbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 22:57:34.223180 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.223098 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a5459afb-0abf-4f18-a271-b6ce127096b6" (UID: "a5459afb-0abf-4f18-a271-b6ce127096b6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 22:57:34.321687 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.321646 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbxbh\" (UniqueName: \"kubernetes.io/projected/a5459afb-0abf-4f18-a271-b6ce127096b6-kube-api-access-vbxbh\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 22:57:34.321687 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.321680 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-serving-cert\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 22:57:34.321687 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.321692 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-service-ca\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 22:57:34.321981 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.321704 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5459afb-0abf-4f18-a271-b6ce127096b6-console-oauth-config\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 22:57:34.321981 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.321716 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-oauth-serving-cert\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 22:57:34.321981 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.321727 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5459afb-0abf-4f18-a271-b6ce127096b6-console-config\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 22:57:34.486770 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.486688 2562 generic.go:356] "Generic (PLEG): container finished" podID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerID="626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0" exitCode=0 Jun 11 22:57:34.486770 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.486732 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b598db68-5hrcp" Jun 11 22:57:34.486770 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.486757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b598db68-5hrcp" event={"ID":"a5459afb-0abf-4f18-a271-b6ce127096b6","Type":"ContainerDied","Data":"626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0"} Jun 11 22:57:34.487002 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.486803 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b598db68-5hrcp" event={"ID":"a5459afb-0abf-4f18-a271-b6ce127096b6","Type":"ContainerDied","Data":"5256b3d0351e9d07575ac96df066968db0f2362ee794bd4df9b0360db67dc5d8"} Jun 11 22:57:34.487002 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.486824 2562 scope.go:117] "RemoveContainer" containerID="626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0" Jun 11 22:57:34.494718 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.494700 2562 scope.go:117] "RemoveContainer" containerID="626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0" Jun 11 22:57:34.494997 ip-10-0-138-236 kubenswrapper[2562]: E0611 22:57:34.494978 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0\": container with ID starting with 626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0 not found: ID does not exist" containerID="626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0" Jun 11 22:57:34.495067 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.495004 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0"} err="failed to get container status \"626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0\": rpc error: code = NotFound desc = could not find container \"626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0\": container with ID starting with 626ea610f770c471dd110a3b75fae6fcfa7daa8c8dbece9c9508353ae386cac0 not found: ID does not exist" Jun 11 22:57:34.508147 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.504806 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84b598db68-5hrcp"] Jun 11 22:57:34.510240 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:34.510221 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84b598db68-5hrcp"] Jun 11 22:57:35.257697 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:35.257664 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:35.459223 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:35.459178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:36.412397 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:36.412366 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" path="/var/lib/kubelet/pods/a5459afb-0abf-4f18-a271-b6ce127096b6/volumes" Jun 11 22:57:36.566293 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:36.566261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:36.621140 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:36.621110 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:36.877582 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:36.877552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:36.953014 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:36.952983 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:40.916240 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:40.916192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:41.014315 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:41.014274 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:44.745206 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:44.745170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:44.812275 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:44.812241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:45.574279 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:45.574242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:55.861901 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:55.861859 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:55.943446 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:55.943404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:56.067005 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:56.066968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:56.124425 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:56.124343 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:57:56.181248 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:57:56.181217 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:58:21.382790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:58:21.382698 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:58:21.448078 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:58:21.448035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:58:50.464484 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:58:50.464448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:58:50.516338 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:58:50.516297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:02.372554 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:02.372509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:02.425605 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:02.425565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:09.599739 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.599706 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-685966d785-6m2jd"] Jun 11 22:59:09.600150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.599971 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" Jun 11 22:59:09.600150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.599982 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" Jun 11 22:59:09.600150 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.600031 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5459afb-0abf-4f18-a271-b6ce127096b6" containerName="console" Jun 11 22:59:09.602879 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.602863 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.606966 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.606943 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 22:59:09.607089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.606947 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 22:59:09.607089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.606997 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 22:59:09.607089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.607002 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 22:59:09.607089 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.607017 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 22:59:09.607424 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.607411 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lr9tq\"" Jun 11 22:59:09.612378 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.612353 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 22:59:09.616708 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.616687 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-685966d785-6m2jd"] Jun 11 22:59:09.657852 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.657820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:09.713081 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.713037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:09.714600 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-trusted-ca-bundle\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.714716 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvqr\" (UniqueName: \"kubernetes.io/projected/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-kube-api-access-njvqr\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.714716 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-oauth-config\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.714716 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714714 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-serving-cert\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.714835 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-config\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.714835 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-oauth-serving-cert\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.714835 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.714766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-service-ca\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.790885 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.790829 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:09.815668 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815639 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-trusted-ca-bundle\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.815790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvqr\" (UniqueName: \"kubernetes.io/projected/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-kube-api-access-njvqr\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.815790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-oauth-config\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.815790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-serving-cert\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.815790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-config\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.815790 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-oauth-serving-cert\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.816306 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.815839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-service-ca\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.816540 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.816517 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-service-ca\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.816653 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.816545 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-config\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.816704 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.816663 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-oauth-serving-cert\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.816746 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.816698 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-trusted-ca-bundle\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.818330 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.818307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-serving-cert\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.818414 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.818351 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-oauth-config\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.823819 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.823798 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvqr\" (UniqueName: \"kubernetes.io/projected/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-kube-api-access-njvqr\") pod \"console-685966d785-6m2jd\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:09.842349 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.842324 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:09.912426 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:09.912371 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:10.039338 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.039314 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-685966d785-6m2jd"] Jun 11 22:59:10.041662 ip-10-0-138-236 kubenswrapper[2562]: W0611 22:59:10.041635 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1d211b_3cb0_4f1f_a34a_b271ed0f0119.slice/crio-ecf3af54bf3de192a551734348286d126e894a7783801eac80625ee8ffb38d6b WatchSource:0}: Error finding container ecf3af54bf3de192a551734348286d126e894a7783801eac80625ee8ffb38d6b: Status 404 returned error can't find the container with id ecf3af54bf3de192a551734348286d126e894a7783801eac80625ee8ffb38d6b Jun 11 22:59:10.405762 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.405733 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:10.454540 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.454510 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:10.820145 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.820109 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685966d785-6m2jd" event={"ID":"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119","Type":"ContainerStarted","Data":"53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715"} Jun 11 22:59:10.820519 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.820161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685966d785-6m2jd" event={"ID":"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119","Type":"ContainerStarted","Data":"ecf3af54bf3de192a551734348286d126e894a7783801eac80625ee8ffb38d6b"} Jun 11 22:59:10.842122 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.842078 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-685966d785-6m2jd" podStartSLOduration=1.84206297 podStartE2EDuration="1.84206297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 22:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 22:59:10.841587103 +0000 UTC m=+402.786835996" watchObservedRunningTime="2026-06-11 22:59:10.84206297 +0000 UTC m=+402.787311864" Jun 11 22:59:10.857324 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.857297 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:10.958114 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:10.958072 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:11.011698 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:11.011661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:19.913595 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:19.913552 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:19.914079 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:19.913621 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:19.919214 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:19.919190 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:20.859377 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:20.859351 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-685966d785-6m2jd" Jun 11 22:59:21.001414 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:21.001379 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:21.054547 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:21.054510 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:21.110900 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:21.110798 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:34.214443 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:34.214405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 22:59:34.307585 ip-10-0-138-236 kubenswrapper[2562]: I0611 22:59:34.307555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:01.305540 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:01.305449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:02.279037 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:02.278996 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:05.061931 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:05.061895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:05.162221 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:05.162183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:09.706346 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:09.706266 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:09.760938 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:09.760902 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:10.960515 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:10.960470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:11.080273 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:11.080242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:11.248362 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:11.248258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:14.573364 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:14.573329 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:14.956968 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:14.956938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:20.517650 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:20.517609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:20.601670 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:20.601640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:22.000766 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:22.000726 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:22.099388 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:22.099356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:22.195961 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:22.195925 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:24.017521 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.017480 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv"] Jun 11 23:00:24.025892 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.025867 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.030047 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.030027 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bdhpl\"" Jun 11 23:00:24.030262 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.030241 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 11 23:00:24.031311 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.031281 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 11 23:00:24.032894 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.032874 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv"] Jun 11 23:00:24.167645 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.167613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtp7f\" (UniqueName: \"kubernetes.io/projected/4f3a2c3e-4e22-4fff-88c1-1b3513d37ece-kube-api-access-qtp7f\") pod \"openshift-lws-operator-fd8ccff4c-h4bsv\" (UID: \"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.167812 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.167656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f3a2c3e-4e22-4fff-88c1-1b3513d37ece-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h4bsv\" (UID: \"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.268998 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.268912 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtp7f\" (UniqueName: \"kubernetes.io/projected/4f3a2c3e-4e22-4fff-88c1-1b3513d37ece-kube-api-access-qtp7f\") pod \"openshift-lws-operator-fd8ccff4c-h4bsv\" (UID: \"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.268998 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.268972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f3a2c3e-4e22-4fff-88c1-1b3513d37ece-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h4bsv\" (UID: \"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.269321 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.269302 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f3a2c3e-4e22-4fff-88c1-1b3513d37ece-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h4bsv\" (UID: \"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.278326 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.278292 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtp7f\" (UniqueName: \"kubernetes.io/projected/4f3a2c3e-4e22-4fff-88c1-1b3513d37ece-kube-api-access-qtp7f\") pod \"openshift-lws-operator-fd8ccff4c-h4bsv\" (UID: \"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.334796 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.334766 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" Jun 11 23:00:24.455779 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.455748 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv"] Jun 11 23:00:24.459397 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:00:24.459371 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f3a2c3e_4e22_4fff_88c1_1b3513d37ece.slice/crio-4fc8c43c71230774958fc231332bdc7d430545a0cb0ea661250ef455ab67b34d WatchSource:0}: Error finding container 4fc8c43c71230774958fc231332bdc7d430545a0cb0ea661250ef455ab67b34d: Status 404 returned error can't find the container with id 4fc8c43c71230774958fc231332bdc7d430545a0cb0ea661250ef455ab67b34d Jun 11 23:00:24.461501 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:24.461486 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 23:00:25.079740 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:25.079706 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" event={"ID":"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece","Type":"ContainerStarted","Data":"4fc8c43c71230774958fc231332bdc7d430545a0cb0ea661250ef455ab67b34d"} Jun 11 23:00:28.095273 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:28.095231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" event={"ID":"4f3a2c3e-4e22-4fff-88c1-1b3513d37ece","Type":"ContainerStarted","Data":"3d1f5a95d59e6d1ff8652d3b492a78d53aeb3971f51a70f5468b058871223c56"} Jun 11 23:00:28.113987 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:28.113937 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h4bsv" podStartSLOduration=1.912498561 podStartE2EDuration="5.113922514s" totalImagesPullingTime="3.201423953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:00:28.112741737 +0000 UTC m=+480.057990628" watchObservedRunningTime="2026-06-11 23:00:28.113922514 +0000 UTC m=+480.059171404" Jun 11 23:00:28.743684 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:28.743643 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:28.856297 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:28.856264 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:30.530464 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:30.530426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:34.287302 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:34.287267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:34.388221 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:34.388181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:42.165576 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:42.165544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:42.267007 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:42.266971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:42.768308 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:42.768267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:42.959369 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:42.959340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.061151 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.061077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.089730 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.089701 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp"] Jun 11 23:00:43.096287 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.096260 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.099592 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.099568 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 23:00:43.099838 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.099821 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 23:00:43.099958 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.099894 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 23:00:43.099958 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.099921 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bv85g\"" Jun 11 23:00:43.100617 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.100602 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 23:00:43.102472 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.102451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96b109cf-c44b-4a83-9130-22a6cc268ff5-manifests\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.102574 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.102498 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96b109cf-c44b-4a83-9130-22a6cc268ff5-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.102574 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.102551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96b109cf-c44b-4a83-9130-22a6cc268ff5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.102684 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.102600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b109cf-c44b-4a83-9130-22a6cc268ff5-tmp\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.102684 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.102658 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/96b109cf-c44b-4a83-9130-22a6cc268ff5-kube-api-access-h9dp6\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.122419 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.122395 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp"] Jun 11 23:00:43.203479 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.203448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96b109cf-c44b-4a83-9130-22a6cc268ff5-manifests\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.203939 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.203491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96b109cf-c44b-4a83-9130-22a6cc268ff5-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.203939 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.203514 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96b109cf-c44b-4a83-9130-22a6cc268ff5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.203939 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.203543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b109cf-c44b-4a83-9130-22a6cc268ff5-tmp\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.203939 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.203575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/96b109cf-c44b-4a83-9130-22a6cc268ff5-kube-api-access-h9dp6\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.204114 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.203966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96b109cf-c44b-4a83-9130-22a6cc268ff5-manifests\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.205883 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.205836 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b109cf-c44b-4a83-9130-22a6cc268ff5-tmp\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.206085 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.206069 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96b109cf-c44b-4a83-9130-22a6cc268ff5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.206128 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.206073 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96b109cf-c44b-4a83-9130-22a6cc268ff5-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.228432 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.228405 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9dp6\" (UniqueName: \"kubernetes.io/projected/96b109cf-c44b-4a83-9130-22a6cc268ff5-kube-api-access-h9dp6\") pod \"opendatahub-operator-controller-manager-74f864d649-2hgvp\" (UID: \"96b109cf-c44b-4a83-9130-22a6cc268ff5\") " pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.290994 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.290964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.391546 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.391464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.405835 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.405811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:43.493011 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.492978 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.549019 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.548983 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp"] Jun 11 23:00:43.552886 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:00:43.552855 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b109cf_c44b_4a83_9130_22a6cc268ff5.slice/crio-d3035a615c5205258202e96c9e8aadaf9fcc2a7155b6fccea508decdaba36c45 WatchSource:0}: Error finding container d3035a615c5205258202e96c9e8aadaf9fcc2a7155b6fccea508decdaba36c45: Status 404 returned error can't find the container with id d3035a615c5205258202e96c9e8aadaf9fcc2a7155b6fccea508decdaba36c45 Jun 11 23:00:43.571968 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.571941 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.632988 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.632952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:43.730551 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:43.730516 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:44.156593 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:44.156510 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" event={"ID":"96b109cf-c44b-4a83-9130-22a6cc268ff5","Type":"ContainerStarted","Data":"d3035a615c5205258202e96c9e8aadaf9fcc2a7155b6fccea508decdaba36c45"} Jun 11 23:00:44.821594 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:44.821553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:44.947140 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:44.947096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:00:47.174884 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:47.174836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" event={"ID":"96b109cf-c44b-4a83-9130-22a6cc268ff5","Type":"ContainerStarted","Data":"9687030c97b8bb81d8298f02e7c10b79c8c3d8173fa38d02475a43ef402c4953"} Jun 11 23:00:48.180504 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:48.180468 2562 generic.go:356] "Generic (PLEG): container finished" podID="96b109cf-c44b-4a83-9130-22a6cc268ff5" containerID="9687030c97b8bb81d8298f02e7c10b79c8c3d8173fa38d02475a43ef402c4953" exitCode=0 Jun 11 23:00:48.180899 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:48.180555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" event={"ID":"96b109cf-c44b-4a83-9130-22a6cc268ff5","Type":"ContainerDied","Data":"9687030c97b8bb81d8298f02e7c10b79c8c3d8173fa38d02475a43ef402c4953"} Jun 11 23:00:49.186206 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:49.186172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" event={"ID":"96b109cf-c44b-4a83-9130-22a6cc268ff5","Type":"ContainerStarted","Data":"e451f178689348d814a25240b0ec1a8601315ad1735bd75e8d43643e2551c98b"} Jun 11 23:00:49.186567 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:49.186313 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:00:49.210946 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:00:49.210901 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" podStartSLOduration=2.069516987 podStartE2EDuration="6.210888199s" totalImagesPullingTime="3.150027389s" totalInitContainerRuntime="991.343823ms" isStatefulPod=false podCreationTimestamp="2026-06-11 23:00:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:00:49.208498642 +0000 UTC m=+501.153747551" watchObservedRunningTime="2026-06-11 23:00:49.210888199 +0000 UTC m=+501.156137089" Jun 11 23:01:00.192392 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:00.192360 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74f864d649-2hgvp" Jun 11 23:01:02.015811 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.015774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:02.788769 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.788735 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c495b869c-6khwv"] Jun 11 23:01:02.796796 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.796772 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.799507 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.799483 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v6gzp\"" Jun 11 23:01:02.800718 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.800699 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 23:01:02.800873 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.800838 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 23:01:02.801029 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.800730 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 23:01:02.801123 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.800699 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 23:01:02.802580 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.802560 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c495b869c-6khwv"] Jun 11 23:01:02.805258 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.805238 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-mxmkn"] Jun 11 23:01:02.808364 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.808347 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:02.810692 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.810676 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cmvfv\"" Jun 11 23:01:02.810819 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.810802 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 11 23:01:02.823166 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.823142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-mxmkn"] Jun 11 23:01:02.855897 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.855871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4226b38a-6127-4aa7-8dc8-4122d4f2b076-tmp\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.856052 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.855908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggx6z\" (UniqueName: \"kubernetes.io/projected/4226b38a-6127-4aa7-8dc8-4122d4f2b076-kube-api-access-ggx6z\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.856052 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.855927 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvswj\" (UniqueName: \"kubernetes.io/projected/0a853ddc-b4a3-4cdf-a408-bb138034e80d-kube-api-access-lvswj\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:02.856052 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.855952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a853ddc-b4a3-4cdf-a408-bb138034e80d-cert\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:02.856052 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.856032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4226b38a-6127-4aa7-8dc8-4122d4f2b076-tls-certs\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.956645 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.956607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4226b38a-6127-4aa7-8dc8-4122d4f2b076-tmp\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.956807 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.956650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx6z\" (UniqueName: \"kubernetes.io/projected/4226b38a-6127-4aa7-8dc8-4122d4f2b076-kube-api-access-ggx6z\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.956807 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.956674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvswj\" (UniqueName: \"kubernetes.io/projected/0a853ddc-b4a3-4cdf-a408-bb138034e80d-kube-api-access-lvswj\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:02.956807 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.956797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a853ddc-b4a3-4cdf-a408-bb138034e80d-cert\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:02.956985 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.956866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4226b38a-6127-4aa7-8dc8-4122d4f2b076-tls-certs\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.956985 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:01:02.956936 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 11 23:01:02.957083 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:01:02.957005 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a853ddc-b4a3-4cdf-a408-bb138034e80d-cert podName:0a853ddc-b4a3-4cdf-a408-bb138034e80d nodeName:}" failed. No retries permitted until 2026-06-11 23:01:03.45698269 +0000 UTC m=+515.402231560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a853ddc-b4a3-4cdf-a408-bb138034e80d-cert") pod "odh-model-controller-6dc94f689f-mxmkn" (UID: "0a853ddc-b4a3-4cdf-a408-bb138034e80d") : secret "odh-model-controller-webhook-cert" not found Jun 11 23:01:02.958951 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.958924 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4226b38a-6127-4aa7-8dc8-4122d4f2b076-tmp\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.959583 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.959560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4226b38a-6127-4aa7-8dc8-4122d4f2b076-tls-certs\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.965451 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.965428 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggx6z\" (UniqueName: \"kubernetes.io/projected/4226b38a-6127-4aa7-8dc8-4122d4f2b076-kube-api-access-ggx6z\") pod \"kube-auth-proxy-5c495b869c-6khwv\" (UID: \"4226b38a-6127-4aa7-8dc8-4122d4f2b076\") " pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:02.965740 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:02.965720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvswj\" (UniqueName: \"kubernetes.io/projected/0a853ddc-b4a3-4cdf-a408-bb138034e80d-kube-api-access-lvswj\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:03.107824 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.107738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" Jun 11 23:01:03.254070 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.254047 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c495b869c-6khwv"] Jun 11 23:01:03.256651 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:03.256613 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4226b38a_6127_4aa7_8dc8_4122d4f2b076.slice/crio-0b15aec2b1c498dd9b8ba1836b343330d2cba102117d53161c5371ee317be24b WatchSource:0}: Error finding container 0b15aec2b1c498dd9b8ba1836b343330d2cba102117d53161c5371ee317be24b: Status 404 returned error can't find the container with id 0b15aec2b1c498dd9b8ba1836b343330d2cba102117d53161c5371ee317be24b Jun 11 23:01:03.461681 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.461645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a853ddc-b4a3-4cdf-a408-bb138034e80d-cert\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:03.462958 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.462937 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-lxw9r"] Jun 11 23:01:03.464094 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.464076 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a853ddc-b4a3-4cdf-a408-bb138034e80d-cert\") pod \"odh-model-controller-6dc94f689f-mxmkn\" (UID: \"0a853ddc-b4a3-4cdf-a408-bb138034e80d\") " pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:03.467997 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.467982 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.470681 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.470660 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 11 23:01:03.477424 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.477399 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-lxw9r"] Jun 11 23:01:03.562062 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.562028 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dv2v\" (UniqueName: \"kubernetes.io/projected/4f36dab2-ccb3-4541-99c2-f997adcf7536-kube-api-access-4dv2v\") pod \"model-serving-api-5dd488fc48-lxw9r\" (UID: \"4f36dab2-ccb3-4541-99c2-f997adcf7536\") " pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.562224 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.562122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36dab2-ccb3-4541-99c2-f997adcf7536-tls-certs\") pod \"model-serving-api-5dd488fc48-lxw9r\" (UID: \"4f36dab2-ccb3-4541-99c2-f997adcf7536\") " pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.662583 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.662553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36dab2-ccb3-4541-99c2-f997adcf7536-tls-certs\") pod \"model-serving-api-5dd488fc48-lxw9r\" (UID: \"4f36dab2-ccb3-4541-99c2-f997adcf7536\") " pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.662748 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.662602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dv2v\" (UniqueName: \"kubernetes.io/projected/4f36dab2-ccb3-4541-99c2-f997adcf7536-kube-api-access-4dv2v\") pod \"model-serving-api-5dd488fc48-lxw9r\" (UID: \"4f36dab2-ccb3-4541-99c2-f997adcf7536\") " pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.665112 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.665077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f36dab2-ccb3-4541-99c2-f997adcf7536-tls-certs\") pod \"model-serving-api-5dd488fc48-lxw9r\" (UID: \"4f36dab2-ccb3-4541-99c2-f997adcf7536\") " pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.671277 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.671247 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dv2v\" (UniqueName: \"kubernetes.io/projected/4f36dab2-ccb3-4541-99c2-f997adcf7536-kube-api-access-4dv2v\") pod \"model-serving-api-5dd488fc48-lxw9r\" (UID: \"4f36dab2-ccb3-4541-99c2-f997adcf7536\") " pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.717045 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.716982 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:03.779507 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.779479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:03.842157 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.842132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-mxmkn"] Jun 11 23:01:03.844606 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:03.844581 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a853ddc_b4a3_4cdf_a408_bb138034e80d.slice/crio-9f21d0e10e87d412c5f14e4905d053535f21c641eda6ec9eb20822c514a80e76 WatchSource:0}: Error finding container 9f21d0e10e87d412c5f14e4905d053535f21c641eda6ec9eb20822c514a80e76: Status 404 returned error can't find the container with id 9f21d0e10e87d412c5f14e4905d053535f21c641eda6ec9eb20822c514a80e76 Jun 11 23:01:03.914034 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:03.914003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-lxw9r"] Jun 11 23:01:03.917355 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:03.917329 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f36dab2_ccb3_4541_99c2_f997adcf7536.slice/crio-3e6cd72c255ae4a83abddd28c49c16b61978be63c9ec86250c74af3bc3cc0163 WatchSource:0}: Error finding container 3e6cd72c255ae4a83abddd28c49c16b61978be63c9ec86250c74af3bc3cc0163: Status 404 returned error can't find the container with id 3e6cd72c255ae4a83abddd28c49c16b61978be63c9ec86250c74af3bc3cc0163 Jun 11 23:01:04.010345 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:04.010266 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:04.248312 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:04.248277 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" event={"ID":"4f36dab2-ccb3-4541-99c2-f997adcf7536","Type":"ContainerStarted","Data":"3e6cd72c255ae4a83abddd28c49c16b61978be63c9ec86250c74af3bc3cc0163"} Jun 11 23:01:04.250188 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:04.250155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" event={"ID":"0a853ddc-b4a3-4cdf-a408-bb138034e80d","Type":"ContainerStarted","Data":"9f21d0e10e87d412c5f14e4905d053535f21c641eda6ec9eb20822c514a80e76"} Jun 11 23:01:04.251523 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:04.251487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" event={"ID":"4226b38a-6127-4aa7-8dc8-4122d4f2b076","Type":"ContainerStarted","Data":"0b15aec2b1c498dd9b8ba1836b343330d2cba102117d53161c5371ee317be24b"} Jun 11 23:01:08.687137 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:08.687099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:09.277814 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.277765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" event={"ID":"4226b38a-6127-4aa7-8dc8-4122d4f2b076","Type":"ContainerStarted","Data":"7ffc5a6a3a24138424ce200a05986ee009dee37d7b823d692a41b0ece47b68a1"} Jun 11 23:01:09.279227 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.279205 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" event={"ID":"4f36dab2-ccb3-4541-99c2-f997adcf7536","Type":"ContainerStarted","Data":"ed1109370b04a796f6492393012c85ed3de6d0e8d48ebdf540d8ec5cd99b1373"} Jun 11 23:01:09.279358 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.279261 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:09.280516 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.280500 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" event={"ID":"0a853ddc-b4a3-4cdf-a408-bb138034e80d","Type":"ContainerStarted","Data":"b5d5e755e84dc00216f433b9fd023fc7dbd524a8d6303c8b3eb59e26a0180242"} Jun 11 23:01:09.280653 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.280642 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:09.305372 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.305326 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c495b869c-6khwv" podStartSLOduration=2.330237924 podStartE2EDuration="7.305313816s" totalImagesPullingTime="4.975075892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:09.303314724 +0000 UTC m=+521.248563634" watchObservedRunningTime="2026-06-11 23:01:09.305313816 +0000 UTC m=+521.250562753" Jun 11 23:01:09.333697 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.333644 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" podStartSLOduration=2.015638439 podStartE2EDuration="6.333629152s" totalImagesPullingTime="4.317990713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:09.332168004 +0000 UTC m=+521.277416921" watchObservedRunningTime="2026-06-11 23:01:09.333629152 +0000 UTC m=+521.278878042" Jun 11 23:01:09.354255 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:09.354201 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" podStartSLOduration=2.955361505 podStartE2EDuration="7.35418102s" totalImagesPullingTime="4.398819515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:09.353036743 +0000 UTC m=+521.298285656" watchObservedRunningTime="2026-06-11 23:01:09.35418102 +0000 UTC m=+521.299429911" Jun 11 23:01:13.338226 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.338164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:13.398209 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.398173 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zfcqp"] Jun 11 23:01:13.401147 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.401124 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.403653 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.403636 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 11 23:01:13.403752 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.403679 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5rl6n\"" Jun 11 23:01:13.415441 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.415417 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg"] Jun 11 23:01:13.418540 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.418515 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:13.419077 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.419054 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zfcqp"] Jun 11 23:01:13.421403 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.421380 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pmxzc\"" Jun 11 23:01:13.421499 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.421488 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 11 23:01:13.433276 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.433250 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg"] Jun 11 23:01:13.438783 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.438756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:13.448702 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.448677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srq4t\" (UniqueName: \"kubernetes.io/projected/6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098-kube-api-access-srq4t\") pod \"kserve-controller-manager-69fc866d76-zfcqp\" (UID: \"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.448796 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.448707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098-cert\") pod \"kserve-controller-manager-69fc866d76-zfcqp\" (UID: \"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.550070 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.550034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f432d-c676-4ef2-8053-770bebc15650-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:13.550242 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.550092 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srq4t\" (UniqueName: \"kubernetes.io/projected/6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098-kube-api-access-srq4t\") pod \"kserve-controller-manager-69fc866d76-zfcqp\" (UID: \"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.550242 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.550111 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwp9c\" (UniqueName: \"kubernetes.io/projected/aa2f432d-c676-4ef2-8053-770bebc15650-kube-api-access-bwp9c\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:13.550242 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.550131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098-cert\") pod \"kserve-controller-manager-69fc866d76-zfcqp\" (UID: \"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.552494 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.552470 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098-cert\") pod \"kserve-controller-manager-69fc866d76-zfcqp\" (UID: \"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.562302 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.562274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srq4t\" (UniqueName: \"kubernetes.io/projected/6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098-kube-api-access-srq4t\") pod \"kserve-controller-manager-69fc866d76-zfcqp\" (UID: \"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.572989 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.572958 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:13.651168 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.651082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f432d-c676-4ef2-8053-770bebc15650-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:13.651168 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.651142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwp9c\" (UniqueName: \"kubernetes.io/projected/aa2f432d-c676-4ef2-8053-770bebc15650-kube-api-access-bwp9c\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:13.651372 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:01:13.651230 2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 11 23:01:13.651372 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:01:13.651302 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2f432d-c676-4ef2-8053-770bebc15650-cert podName:aa2f432d-c676-4ef2-8053-770bebc15650 nodeName:}" failed. No retries permitted until 2026-06-11 23:01:14.151286675 +0000 UTC m=+526.096535544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/aa2f432d-c676-4ef2-8053-770bebc15650-cert") pod "llmisvc-controller-manager-864bfd7bc9-w7hzg" (UID: "aa2f432d-c676-4ef2-8053-770bebc15650") : secret "llmisvc-webhook-server-cert" not found Jun 11 23:01:13.690090 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.690060 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwp9c\" (UniqueName: \"kubernetes.io/projected/aa2f432d-c676-4ef2-8053-770bebc15650-kube-api-access-bwp9c\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:13.701956 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.701933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:13.710814 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.710794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:13.803694 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.803658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:13.842857 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:13.842811 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zfcqp"] Jun 11 23:01:13.846709 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:13.846685 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e29b810_7f3c_4bb7_a99a_8d4b4e6eb098.slice/crio-71a158b265f571a5497c62ee1821872d2be5799c44b1bf1e5084a78854fab80d WatchSource:0}: Error finding container 71a158b265f571a5497c62ee1821872d2be5799c44b1bf1e5084a78854fab80d: Status 404 returned error can't find the container with id 71a158b265f571a5497c62ee1821872d2be5799c44b1bf1e5084a78854fab80d Jun 11 23:01:14.155669 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:14.155636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f432d-c676-4ef2-8053-770bebc15650-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:14.158075 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:14.158059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2f432d-c676-4ef2-8053-770bebc15650-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w7hzg\" (UID: \"aa2f432d-c676-4ef2-8053-770bebc15650\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:14.202065 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:14.202031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:14.316380 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:14.316338 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" event={"ID":"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098","Type":"ContainerStarted","Data":"71a158b265f571a5497c62ee1821872d2be5799c44b1bf1e5084a78854fab80d"} Jun 11 23:01:14.328060 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:14.328035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:14.461775 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:14.461709 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg"] Jun 11 23:01:14.463808 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:14.463785 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaa2f432d_c676_4ef2_8053_770bebc15650.slice/crio-2e5c1cff8e53678bf1a4c22de925bcfd3d69a74ae7bfa21e048551f53c21e6b1 WatchSource:0}: Error finding container 2e5c1cff8e53678bf1a4c22de925bcfd3d69a74ae7bfa21e048551f53c21e6b1: Status 404 returned error can't find the container with id 2e5c1cff8e53678bf1a4c22de925bcfd3d69a74ae7bfa21e048551f53c21e6b1 Jun 11 23:01:15.321607 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:15.321573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" event={"ID":"aa2f432d-c676-4ef2-8053-770bebc15650","Type":"ContainerStarted","Data":"2e5c1cff8e53678bf1a4c22de925bcfd3d69a74ae7bfa21e048551f53c21e6b1"} Jun 11 23:01:15.951260 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:15.951225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:16.099164 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:16.099074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:16.326960 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:16.326926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" event={"ID":"6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098","Type":"ContainerStarted","Data":"c300b39a18b7bff2f705900784220407926e6717ca7632860e93be4ace41c336"} Jun 11 23:01:16.327106 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:16.327047 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:16.380042 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:16.379992 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" podStartSLOduration=1.012508919 podStartE2EDuration="3.379976712s" totalImagesPullingTime="2.367467793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:16.373421861 +0000 UTC m=+528.318670753" watchObservedRunningTime="2026-06-11 23:01:16.379976712 +0000 UTC m=+528.325225603" Jun 11 23:01:17.861918 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:17.861808 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:17.970456 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:17.970422 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:18.108265 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:18.108221 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:18.241134 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:18.241094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:18.380275 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:18.380240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:19.167695 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.167618 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-784f85db5-xpwpm"] Jun 11 23:01:19.171062 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.171042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.174466 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.174440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2sth9\"" Jun 11 23:01:19.174575 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.174502 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 11 23:01:19.193171 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.193146 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-784f85db5-xpwpm"] Jun 11 23:01:19.228391 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.228363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:19.308996 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.308964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/9d75a68a-0e53-4313-b1b2-a615a90457bb-redirect-config\") pod \"dashboard-redirect-784f85db5-xpwpm\" (UID: \"9d75a68a-0e53-4313-b1b2-a615a90457bb\") " pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.308996 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.308998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wjz7\" (UniqueName: \"kubernetes.io/projected/9d75a68a-0e53-4313-b1b2-a615a90457bb-kube-api-access-7wjz7\") pod \"dashboard-redirect-784f85db5-xpwpm\" (UID: \"9d75a68a-0e53-4313-b1b2-a615a90457bb\") " pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.343246 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.343215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" event={"ID":"aa2f432d-c676-4ef2-8053-770bebc15650","Type":"ContainerStarted","Data":"94fd0a64340eadcaf4b231ae16d99118745a4d8ab9264654b147cd94c86263d1"} Jun 11 23:01:19.343422 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.343325 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:01:19.369780 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.369732 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" podStartSLOduration=2.114259211 podStartE2EDuration="6.369717838s" totalImagesPullingTime="4.255458627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:19.368117493 +0000 UTC m=+531.313366419" watchObservedRunningTime="2026-06-11 23:01:19.369717838 +0000 UTC m=+531.314966770" Jun 11 23:01:19.410406 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.410381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/9d75a68a-0e53-4313-b1b2-a615a90457bb-redirect-config\") pod \"dashboard-redirect-784f85db5-xpwpm\" (UID: \"9d75a68a-0e53-4313-b1b2-a615a90457bb\") " pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.410406 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.410408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wjz7\" (UniqueName: \"kubernetes.io/projected/9d75a68a-0e53-4313-b1b2-a615a90457bb-kube-api-access-7wjz7\") pod \"dashboard-redirect-784f85db5-xpwpm\" (UID: \"9d75a68a-0e53-4313-b1b2-a615a90457bb\") " pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.411282 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.411262 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/9d75a68a-0e53-4313-b1b2-a615a90457bb-redirect-config\") pod \"dashboard-redirect-784f85db5-xpwpm\" (UID: \"9d75a68a-0e53-4313-b1b2-a615a90457bb\") " pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.425766 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.425698 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wjz7\" (UniqueName: \"kubernetes.io/projected/9d75a68a-0e53-4313-b1b2-a615a90457bb-kube-api-access-7wjz7\") pod \"dashboard-redirect-784f85db5-xpwpm\" (UID: \"9d75a68a-0e53-4313-b1b2-a615a90457bb\") " pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.481619 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.481593 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" Jun 11 23:01:19.629956 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.629930 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-784f85db5-xpwpm"] Jun 11 23:01:19.633039 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:19.633011 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d75a68a_0e53_4313_b1b2_a615a90457bb.slice/crio-76a9607bd65c916cda8d5778b7f304d217d4d07e5496491dfce25d0bcf007e78 WatchSource:0}: Error finding container 76a9607bd65c916cda8d5778b7f304d217d4d07e5496491dfce25d0bcf007e78: Status 404 returned error can't find the container with id 76a9607bd65c916cda8d5778b7f304d217d4d07e5496491dfce25d0bcf007e78 Jun 11 23:01:19.670784 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.670756 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-7577f"] Jun 11 23:01:19.675137 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.675119 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.680373 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.680353 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 11 23:01:19.680916 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.680896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-m8f46\"" Jun 11 23:01:19.681427 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.681408 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 11 23:01:19.708812 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.708785 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-7577f"] Jun 11 23:01:19.814039 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.814001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv99k\" (UniqueName: \"kubernetes.io/projected/6085e851-2d24-4956-93ff-10a1da9d8944-kube-api-access-pv99k\") pod \"servicemesh-operator3-57f65f65fb-7577f\" (UID: \"6085e851-2d24-4956-93ff-10a1da9d8944\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.814239 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.814176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/6085e851-2d24-4956-93ff-10a1da9d8944-operator-config\") pod \"servicemesh-operator3-57f65f65fb-7577f\" (UID: \"6085e851-2d24-4956-93ff-10a1da9d8944\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.915229 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.915189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/6085e851-2d24-4956-93ff-10a1da9d8944-operator-config\") pod \"servicemesh-operator3-57f65f65fb-7577f\" (UID: \"6085e851-2d24-4956-93ff-10a1da9d8944\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.915405 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.915259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv99k\" (UniqueName: \"kubernetes.io/projected/6085e851-2d24-4956-93ff-10a1da9d8944-kube-api-access-pv99k\") pod \"servicemesh-operator3-57f65f65fb-7577f\" (UID: \"6085e851-2d24-4956-93ff-10a1da9d8944\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.918492 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.918465 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/6085e851-2d24-4956-93ff-10a1da9d8944-operator-config\") pod \"servicemesh-operator3-57f65f65fb-7577f\" (UID: \"6085e851-2d24-4956-93ff-10a1da9d8944\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.928710 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.928687 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv99k\" (UniqueName: \"kubernetes.io/projected/6085e851-2d24-4956-93ff-10a1da9d8944-kube-api-access-pv99k\") pod \"servicemesh-operator3-57f65f65fb-7577f\" (UID: \"6085e851-2d24-4956-93ff-10a1da9d8944\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:19.984365 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:19.984306 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:20.144559 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:20.144536 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-7577f"] Jun 11 23:01:20.147063 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:20.147036 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6085e851_2d24_4956_93ff_10a1da9d8944.slice/crio-888a6b677a9867bf7a966357a09d343f758938139d4a4fb5545b8b3ca3b6325d WatchSource:0}: Error finding container 888a6b677a9867bf7a966357a09d343f758938139d4a4fb5545b8b3ca3b6325d: Status 404 returned error can't find the container with id 888a6b677a9867bf7a966357a09d343f758938139d4a4fb5545b8b3ca3b6325d Jun 11 23:01:20.286682 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:20.286594 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-mxmkn" Jun 11 23:01:20.288650 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:20.288629 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-lxw9r" Jun 11 23:01:20.348496 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:20.348459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" event={"ID":"9d75a68a-0e53-4313-b1b2-a615a90457bb","Type":"ContainerStarted","Data":"76a9607bd65c916cda8d5778b7f304d217d4d07e5496491dfce25d0bcf007e78"} Jun 11 23:01:20.349921 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:20.349896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" event={"ID":"6085e851-2d24-4956-93ff-10a1da9d8944","Type":"ContainerStarted","Data":"888a6b677a9867bf7a966357a09d343f758938139d4a4fb5545b8b3ca3b6325d"} Jun 11 23:01:22.363514 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:22.363478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" event={"ID":"9d75a68a-0e53-4313-b1b2-a615a90457bb","Type":"ContainerStarted","Data":"55d704541b81b308cccf9b679bd71c9e4528306f68e44123c77a716f103818d2"} Jun 11 23:01:22.409369 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:22.409200 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-784f85db5-xpwpm" podStartSLOduration=1.499873915 podStartE2EDuration="3.409179342s" totalImagesPullingTime="1.909305427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:22.405492975 +0000 UTC m=+534.350741900" watchObservedRunningTime="2026-06-11 23:01:22.409179342 +0000 UTC m=+534.354428233" Jun 11 23:01:24.004408 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:24.004370 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:24.375179 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:24.375094 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" event={"ID":"6085e851-2d24-4956-93ff-10a1da9d8944","Type":"ContainerStarted","Data":"d62783af751c018d8bf213ba4feea3bedece4235d6bda0c7e943d620c679b51e"} Jun 11 23:01:24.375346 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:24.375247 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:24.398420 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:24.398364 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" podStartSLOduration=1.511378361 podStartE2EDuration="5.398346227s" totalImagesPullingTime="3.886967866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:24.396417116 +0000 UTC m=+536.341666007" watchObservedRunningTime="2026-06-11 23:01:24.398346227 +0000 UTC m=+536.343595120" Jun 11 23:01:26.057293 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:26.057258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:26.158926 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:26.158894 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:28.625774 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.625712 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m"] Jun 11 23:01:28.630739 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.630718 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.633380 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.633344 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 11 23:01:28.633380 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.633363 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 11 23:01:28.633519 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.633360 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 11 23:01:28.633695 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.633668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:28.633797 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.633729 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 11 23:01:28.633891 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.633798 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qwkfm\"" Jun 11 23:01:28.641324 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.641304 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m"] Jun 11 23:01:28.779946 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.779911 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:28.793096 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmz5w\" (UniqueName: \"kubernetes.io/projected/9994305c-64de-4b66-a50b-041b96c1d7a8-kube-api-access-qmz5w\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.793176 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.793176 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.793176 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.793284 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.793284 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793217 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9994305c-64de-4b66-a50b-041b96c1d7a8-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.793284 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.793259 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.862012 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.861976 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:28.893702 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmz5w\" (UniqueName: \"kubernetes.io/projected/9994305c-64de-4b66-a50b-041b96c1d7a8-kube-api-access-qmz5w\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.893702 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.893702 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.893953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.893953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.893953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9994305c-64de-4b66-a50b-041b96c1d7a8-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.894089 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.893982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.894487 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.894457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.896144 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.896117 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.896257 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.896119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9994305c-64de-4b66-a50b-041b96c1d7a8-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.896558 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.896538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.896619 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.896538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9994305c-64de-4b66-a50b-041b96c1d7a8-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.904476 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.904440 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmz5w\" (UniqueName: \"kubernetes.io/projected/9994305c-64de-4b66-a50b-041b96c1d7a8-kube-api-access-qmz5w\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.904597 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.904486 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9994305c-64de-4b66-a50b-041b96c1d7a8-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-bfm8m\" (UID: \"9994305c-64de-4b66-a50b-041b96c1d7a8\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.941056 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.941034 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:28.960089 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:28.960053 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:29.077605 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:29.077538 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m"] Jun 11 23:01:29.080223 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:01:29.080186 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9994305c_64de_4b66_a50b_041b96c1d7a8.slice/crio-65a22b49d102d23cb2851142363b8ef9955a8086d26ee4a5af65f25cc932f77e WatchSource:0}: Error finding container 65a22b49d102d23cb2851142363b8ef9955a8086d26ee4a5af65f25cc932f77e: Status 404 returned error can't find the container with id 65a22b49d102d23cb2851142363b8ef9955a8086d26ee4a5af65f25cc932f77e Jun 11 23:01:29.268166 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:29.268130 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:29.397477 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:29.397444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" event={"ID":"9994305c-64de-4b66-a50b-041b96c1d7a8","Type":"ContainerStarted","Data":"65a22b49d102d23cb2851142363b8ef9955a8086d26ee4a5af65f25cc932f77e"} Jun 11 23:01:33.056795 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:33.056759 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 11 23:01:33.057037 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:33.056824 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 11 23:01:33.419373 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:33.419286 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" event={"ID":"9994305c-64de-4b66-a50b-041b96c1d7a8","Type":"ContainerStarted","Data":"3f483832ffc0d7281a57f8cb21649c3c265b3b47abf2bdfb96fddcb7b3543d10"} Jun 11 23:01:33.419512 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:33.419383 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:33.439324 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:33.439279 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" podStartSLOduration=1.465228252 podStartE2EDuration="5.439266186s" totalImagesPullingTime="3.974037934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:01:33.438545413 +0000 UTC m=+545.383794307" watchObservedRunningTime="2026-06-11 23:01:33.439266186 +0000 UTC m=+545.384515076" Jun 11 23:01:34.067641 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.067597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:34.167533 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.167499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:34.363242 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.363155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:34.425518 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.425487 2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-bfm8m container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 11 23:01:34.425683 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.425548 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" podUID="9994305c-64de-4b66-a50b-041b96c1d7a8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 23:01:34.540086 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.540052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:34.633574 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:34.633498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:35.109590 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:35.109551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:35.202806 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:35.202771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:35.381884 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:35.381786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-7577f" Jun 11 23:01:37.424876 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:37.424826 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-bfm8m" Jun 11 23:01:38.379337 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:38.379299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:01:47.336773 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:47.336736 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-zfcqp" Jun 11 23:01:50.352678 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:01:50.352644 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w7hzg" Jun 11 23:02:03.578214 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:03.578176 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:03.676240 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:03.676194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:03.757033 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:03.756997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:04.498877 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:04.498824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:04.700492 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:04.700459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:17.719092 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:17.719058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:28.403139 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:28.403111 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 23:02:28.403798 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:28.403777 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 23:02:32.445414 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:32.445374 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:33.149301 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:33.149253 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:34.688496 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:34.688461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:39.874937 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:39.874903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:43.716864 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:43.716818 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:45.698779 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.698742 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp"] Jun 11 23:02:45.701960 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.701942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:45.712418 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.712399 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 23:02:45.712922 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.712905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 11 23:02:45.713507 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.713494 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dg4jn\"" Jun 11 23:02:45.721706 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.721688 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 23:02:45.738861 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.736391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qznlm\" (UniqueName: \"kubernetes.io/projected/2d80928c-ccf1-4def-b144-d6680ffe7182-kube-api-access-qznlm\") pod \"dns-operator-controller-manager-864dd78c4c-9zlcp\" (UID: \"2d80928c-ccf1-4def-b144-d6680ffe7182\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:45.742776 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.741436 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp"] Jun 11 23:02:45.837528 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.837493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qznlm\" (UniqueName: \"kubernetes.io/projected/2d80928c-ccf1-4def-b144-d6680ffe7182-kube-api-access-qznlm\") pod \"dns-operator-controller-manager-864dd78c4c-9zlcp\" (UID: \"2d80928c-ccf1-4def-b144-d6680ffe7182\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:45.856944 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:45.856912 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qznlm\" (UniqueName: \"kubernetes.io/projected/2d80928c-ccf1-4def-b144-d6680ffe7182-kube-api-access-qznlm\") pod \"dns-operator-controller-manager-864dd78c4c-9zlcp\" (UID: \"2d80928c-ccf1-4def-b144-d6680ffe7182\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:46.012288 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:46.012200 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:46.170190 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:46.170160 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp"] Jun 11 23:02:46.171157 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:02:46.171127 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d80928c_ccf1_4def_b144_d6680ffe7182.slice/crio-d365a9b42421123a1ed4b1500b8c56cef8bbae094036af4795707e5c70c1b218 WatchSource:0}: Error finding container d365a9b42421123a1ed4b1500b8c56cef8bbae094036af4795707e5c70c1b218: Status 404 returned error can't find the container with id d365a9b42421123a1ed4b1500b8c56cef8bbae094036af4795707e5c70c1b218 Jun 11 23:02:46.494542 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:46.494513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:46.745751 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:46.745682 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" event={"ID":"2d80928c-ccf1-4def-b144-d6680ffe7182","Type":"ContainerStarted","Data":"d365a9b42421123a1ed4b1500b8c56cef8bbae094036af4795707e5c70c1b218"} Jun 11 23:02:47.751775 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.751746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" event={"ID":"2d80928c-ccf1-4def-b144-d6680ffe7182","Type":"ContainerStarted","Data":"2145fcce5929de7a13d5e74de4676a711b4cfeef9dfc14746e522fb01a18e468"} Jun 11 23:02:47.752159 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.751883 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:47.792018 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.791967 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" podStartSLOduration=1.324526845 podStartE2EDuration="2.791953295s" totalImagesPullingTime="1.46742645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:02:47.789546072 +0000 UTC m=+619.734794973" watchObservedRunningTime="2026-06-11 23:02:47.791953295 +0000 UTC m=+619.737202186" Jun 11 23:02:47.896634 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.896563 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-98f8597f5-9glwm"] Jun 11 23:02:47.899867 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.899832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955582 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955555 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-config\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955732 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-oauth-config\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955732 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-serving-cert\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955732 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phdv5\" (UniqueName: \"kubernetes.io/projected/7f6663ef-71bc-487f-b572-77e8d524cdbf-kube-api-access-phdv5\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955890 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-service-ca\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955890 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-trusted-ca-bundle\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.955890 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.955886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-oauth-serving-cert\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:47.962367 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:47.962342 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98f8597f5-9glwm"] Jun 11 23:02:48.056779 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.056745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-oauth-serving-cert\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.056779 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.056780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-config\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057011 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.056802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-oauth-config\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057011 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.056935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-serving-cert\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057011 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.056974 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phdv5\" (UniqueName: \"kubernetes.io/projected/7f6663ef-71bc-487f-b572-77e8d524cdbf-kube-api-access-phdv5\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057127 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.057026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-service-ca\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057127 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.057075 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-trusted-ca-bundle\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057520 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.057491 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-config\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057668 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.057646 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-oauth-serving-cert\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.057821 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.057797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-service-ca\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.058079 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.058062 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f6663ef-71bc-487f-b572-77e8d524cdbf-trusted-ca-bundle\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.059316 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.059295 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-serving-cert\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.059402 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.059323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6663ef-71bc-487f-b572-77e8d524cdbf-console-oauth-config\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.074814 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.074789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:48.076292 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.076268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phdv5\" (UniqueName: \"kubernetes.io/projected/7f6663ef-71bc-487f-b572-77e8d524cdbf-kube-api-access-phdv5\") pod \"console-98f8597f5-9glwm\" (UID: \"7f6663ef-71bc-487f-b572-77e8d524cdbf\") " pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.209201 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.209165 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:48.302535 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.302499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:48.350456 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.350431 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98f8597f5-9glwm"] Jun 11 23:02:48.353093 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:02:48.353064 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6663ef_71bc_487f_b572_77e8d524cdbf.slice/crio-2dcb08f2c0570c13fcd468bcf1b6aec0087071a1cc30028c8c0be12d65dd5638 WatchSource:0}: Error finding container 2dcb08f2c0570c13fcd468bcf1b6aec0087071a1cc30028c8c0be12d65dd5638: Status 404 returned error can't find the container with id 2dcb08f2c0570c13fcd468bcf1b6aec0087071a1cc30028c8c0be12d65dd5638 Jun 11 23:02:48.758108 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.758070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98f8597f5-9glwm" event={"ID":"7f6663ef-71bc-487f-b572-77e8d524cdbf","Type":"ContainerStarted","Data":"78f9edb2222445917fe0c1d2e5bb7020e268369a3a22df0ba4ae142d9dbc89b6"} Jun 11 23:02:48.758476 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.758128 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98f8597f5-9glwm" event={"ID":"7f6663ef-71bc-487f-b572-77e8d524cdbf","Type":"ContainerStarted","Data":"2dcb08f2c0570c13fcd468bcf1b6aec0087071a1cc30028c8c0be12d65dd5638"} Jun 11 23:02:48.803476 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:48.803426 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-98f8597f5-9glwm" podStartSLOduration=1.803411993 podStartE2EDuration="1.803411993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:02:48.802648271 +0000 UTC m=+620.747897181" watchObservedRunningTime="2026-06-11 23:02:48.803411993 +0000 UTC m=+620.748660884" Jun 11 23:02:49.119092 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:49.119004 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:50.707355 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.707316 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm"] Jun 11 23:02:50.710777 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.710762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:50.715378 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.715358 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fhlt6\"" Jun 11 23:02:50.732290 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.732260 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm"] Jun 11 23:02:50.781860 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.781819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:50.782019 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.781914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw2r9\" (UniqueName: \"kubernetes.io/projected/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-kube-api-access-tw2r9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:50.849424 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.849393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:50.883021 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.882993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:50.883192 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.883053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw2r9\" (UniqueName: \"kubernetes.io/projected/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-kube-api-access-tw2r9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:50.883389 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.883368 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:50.918686 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:50.918658 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw2r9\" (UniqueName: \"kubernetes.io/projected/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-kube-api-access-tw2r9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:51.021148 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:51.021060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:51.155711 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:51.155684 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm"] Jun 11 23:02:51.158705 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:02:51.158677 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77c917b_90b5_44a4_bc39_4d43c23b0fb4.slice/crio-b56caae10d61f13019d4a5f6b76c9e1037cd06a4cf0bc04449997f5b697816ba WatchSource:0}: Error finding container b56caae10d61f13019d4a5f6b76c9e1037cd06a4cf0bc04449997f5b697816ba: Status 404 returned error can't find the container with id b56caae10d61f13019d4a5f6b76c9e1037cd06a4cf0bc04449997f5b697816ba Jun 11 23:02:51.772790 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:51.772753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" event={"ID":"a77c917b-90b5-44a4-bc39-4d43c23b0fb4","Type":"ContainerStarted","Data":"b56caae10d61f13019d4a5f6b76c9e1037cd06a4cf0bc04449997f5b697816ba"} Jun 11 23:02:52.477609 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:52.477577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:52.586099 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:52.586058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:52.690675 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:52.690640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:52.909109 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:52.909026 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:53.005265 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:53.005236 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:56.356411 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:56.356374 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:56.529036 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:56.528993 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:57.804401 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:57.804369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" event={"ID":"a77c917b-90b5-44a4-bc39-4d43c23b0fb4","Type":"ContainerStarted","Data":"054674ee426f55722b56c690b646317ebe4e040d1e291d99f8becf5161b641e9"} Jun 11 23:02:57.804786 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:57.804623 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:02:57.864271 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:57.864220 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" podStartSLOduration=1.385628484 podStartE2EDuration="7.864197585s" totalImagesPullingTime="6.478569101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:02:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:02:57.86259055 +0000 UTC m=+629.807839453" watchObservedRunningTime="2026-06-11 23:02:57.864197585 +0000 UTC m=+629.809446475" Jun 11 23:02:58.209694 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.209647 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:58.209930 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.209711 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:58.215701 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.215676 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:58.365357 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.365323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:58.460976 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.460886 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:58.760663 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.760576 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9zlcp" Jun 11 23:02:58.817648 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.817624 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-98f8597f5-9glwm" Jun 11 23:02:58.910725 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:58.910689 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-685966d785-6m2jd"] Jun 11 23:02:59.108627 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:59.108541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:02:59.263516 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:02:59.263482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:00.719488 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:00.719446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:01.002813 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:01.002725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:03.557543 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:03.557509 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:03.644598 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:03.644567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:08.815088 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:08.815058 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:09.476390 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:09.476354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:09.608376 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:09.608339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:10.732261 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.732229 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm"] Jun 11 23:03:10.732706 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.732402 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" containerName="manager" containerID="cri-o://054674ee426f55722b56c690b646317ebe4e040d1e291d99f8becf5161b641e9" gracePeriod=2 Jun 11 23:03:10.756283 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.756257 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm"] Jun 11 23:03:10.768895 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:10.768837 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\": pods \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:10.774989 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.774964 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45"] Jun 11 23:03:10.775369 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.775350 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" containerName="manager" Jun 11 23:03:10.775369 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.775368 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" containerName="manager" Jun 11 23:03:10.775478 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.775426 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" containerName="manager" Jun 11 23:03:10.778292 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.778269 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:10.805953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.805921 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45"] Jun 11 23:03:10.845591 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:10.845552 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" is forbidden: User \"system:node:ip-10-0-138-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-236.ec2.internal' and this object" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:10.871797 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.871770 2562 generic.go:356] "Generic (PLEG): container finished" podID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" containerID="054674ee426f55722b56c690b646317ebe4e040d1e291d99f8becf5161b641e9" exitCode=0 Jun 11 23:03:10.966122 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.966092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92xq\" (UniqueName: \"kubernetes.io/projected/7586a387-706b-4d33-9ad1-088616c7ac89-kube-api-access-x92xq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95s45\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:10.966255 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.966149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586a387-706b-4d33-9ad1-088616c7ac89-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95s45\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:10.984091 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:10.984036 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:10.986782 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:10.986760 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" is forbidden: User \"system:node:ip-10-0-138-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-236.ec2.internal' and this object" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:11.067603 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.067570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586a387-706b-4d33-9ad1-088616c7ac89-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95s45\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:11.067770 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.067651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x92xq\" (UniqueName: \"kubernetes.io/projected/7586a387-706b-4d33-9ad1-088616c7ac89-kube-api-access-x92xq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95s45\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:11.067991 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.067971 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586a387-706b-4d33-9ad1-088616c7ac89-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95s45\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:11.079913 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.079887 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92xq\" (UniqueName: \"kubernetes.io/projected/7586a387-706b-4d33-9ad1-088616c7ac89-kube-api-access-x92xq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95s45\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:11.115203 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.115176 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:11.168719 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.168686 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw2r9\" (UniqueName: \"kubernetes.io/projected/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-kube-api-access-tw2r9\") pod \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " Jun 11 23:03:11.168882 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.168787 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-extensions-socket-volume\") pod \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\" (UID: \"a77c917b-90b5-44a4-bc39-4d43c23b0fb4\") " Jun 11 23:03:11.169407 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.169370 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a77c917b-90b5-44a4-bc39-4d43c23b0fb4" (UID: "a77c917b-90b5-44a4-bc39-4d43c23b0fb4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 23:03:11.171540 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.171507 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-kube-api-access-tw2r9" (OuterVolumeSpecName: "kube-api-access-tw2r9") pod "a77c917b-90b5-44a4-bc39-4d43c23b0fb4" (UID: "a77c917b-90b5-44a4-bc39-4d43c23b0fb4"). InnerVolumeSpecName "kube-api-access-tw2r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 23:03:11.263820 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.263794 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45"] Jun 11 23:03:11.265834 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:03:11.265803 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7586a387_706b_4d33_9ad1_088616c7ac89.slice/crio-b0fb741baa9f656eddd9fb43170dac96e1a3fd16ecd4b186f616e6a2188a6148 WatchSource:0}: Error finding container b0fb741baa9f656eddd9fb43170dac96e1a3fd16ecd4b186f616e6a2188a6148: Status 404 returned error can't find the container with id b0fb741baa9f656eddd9fb43170dac96e1a3fd16ecd4b186f616e6a2188a6148 Jun 11 23:03:11.270581 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.270556 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw2r9\" (UniqueName: \"kubernetes.io/projected/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-kube-api-access-tw2r9\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:11.270659 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.270586 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a77c917b-90b5-44a4-bc39-4d43c23b0fb4-extensions-socket-volume\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:11.878479 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.878451 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:11.878479 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.878466 2562 scope.go:117] "RemoveContainer" containerID="054674ee426f55722b56c690b646317ebe4e040d1e291d99f8becf5161b641e9" Jun 11 23:03:11.880398 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.880371 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" event={"ID":"7586a387-706b-4d33-9ad1-088616c7ac89","Type":"ContainerStarted","Data":"6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef"} Jun 11 23:03:11.880490 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.880424 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" event={"ID":"7586a387-706b-4d33-9ad1-088616c7ac89","Type":"ContainerStarted","Data":"b0fb741baa9f656eddd9fb43170dac96e1a3fd16ecd4b186f616e6a2188a6148"} Jun 11 23:03:11.880490 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.880457 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:11.881305 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:11.881276 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" is forbidden: User \"system:node:ip-10-0-138-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-236.ec2.internal' and this object" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:11.883401 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:11.883378 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" is forbidden: User \"system:node:ip-10-0-138-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-236.ec2.internal' and this object" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:11.907124 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:11.907089 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c7xcm\" is forbidden: User \"system:node:ip-10-0-138-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-236.ec2.internal' and this object" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c7xcm" Jun 11 23:03:11.907570 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:11.907520 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" podStartSLOduration=1.9075017490000001 podStartE2EDuration="1.907501749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:03:11.904459034 +0000 UTC m=+643.849707953" watchObservedRunningTime="2026-06-11 23:03:11.907501749 +0000 UTC m=+643.852750643" Jun 11 23:03:12.414250 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:12.414218 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a77c917b-90b5-44a4-bc39-4d43c23b0fb4" path="/var/lib/kubelet/pods/a77c917b-90b5-44a4-bc39-4d43c23b0fb4/volumes" Jun 11 23:03:13.879290 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:13.879258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:13.980067 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:13.980014 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:14.871226 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:14.871196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:15.010175 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:15.010146 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:19.091548 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:19.091513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:19.188607 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:19.188569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:22.066347 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:22.066304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:22.167648 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:22.167614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:22.888693 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:22.888661 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:23.930310 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:23.930274 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-685966d785-6m2jd" podUID="6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" containerName="console" containerID="cri-o://53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715" gracePeriod=15 Jun 11 23:03:24.188354 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.188300 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685966d785-6m2jd" Jun 11 23:03:24.270113 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270083 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njvqr\" (UniqueName: \"kubernetes.io/projected/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-kube-api-access-njvqr\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270113 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270118 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-serving-cert\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270342 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270143 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-oauth-serving-cert\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270342 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270177 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-oauth-config\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270342 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270215 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-trusted-ca-bundle\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270342 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270237 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-service-ca\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270542 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270399 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-config\") pod \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\" (UID: \"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119\") " Jun 11 23:03:24.270542 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270506 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 23:03:24.270772 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270747 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 23:03:24.270907 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270766 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 23:03:24.270907 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270883 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-config" (OuterVolumeSpecName: "console-config") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 23:03:24.271015 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270914 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-trusted-ca-bundle\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.271015 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270934 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-service-ca\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.271015 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.270946 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-oauth-serving-cert\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.272405 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.272382 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 23:03:24.272485 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.272401 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 23:03:24.272485 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.272460 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-kube-api-access-njvqr" (OuterVolumeSpecName: "kube-api-access-njvqr") pod "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" (UID: "6d1d211b-3cb0-4f1f-a34a-b271ed0f0119"). InnerVolumeSpecName "kube-api-access-njvqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 23:03:24.371910 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.371882 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njvqr\" (UniqueName: \"kubernetes.io/projected/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-kube-api-access-njvqr\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.371910 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.371903 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-serving-cert\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.372079 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.371924 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-oauth-config\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.372079 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.371932 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119-console-config\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:24.946699 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.946660 2562 generic.go:356] "Generic (PLEG): container finished" podID="6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" containerID="53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715" exitCode=0 Jun 11 23:03:24.946699 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.946703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685966d785-6m2jd" event={"ID":"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119","Type":"ContainerDied","Data":"53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715"} Jun 11 23:03:24.947226 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.946714 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685966d785-6m2jd" Jun 11 23:03:24.947226 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.946749 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685966d785-6m2jd" event={"ID":"6d1d211b-3cb0-4f1f-a34a-b271ed0f0119","Type":"ContainerDied","Data":"ecf3af54bf3de192a551734348286d126e894a7783801eac80625ee8ffb38d6b"} Jun 11 23:03:24.947226 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.946770 2562 scope.go:117] "RemoveContainer" containerID="53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715" Jun 11 23:03:24.955250 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.955234 2562 scope.go:117] "RemoveContainer" containerID="53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715" Jun 11 23:03:24.955511 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:24.955490 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715\": container with ID starting with 53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715 not found: ID does not exist" containerID="53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715" Jun 11 23:03:24.955571 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.955524 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715"} err="failed to get container status \"53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715\": rpc error: code = NotFound desc = could not find container \"53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715\": container with ID starting with 53ecb6febdd22385e1c03d147579c96ef55949e15b28a9dcbd09a049d8133715 not found: ID does not exist" Jun 11 23:03:24.968719 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.968693 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-685966d785-6m2jd"] Jun 11 23:03:24.982616 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:24.982589 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-685966d785-6m2jd"] Jun 11 23:03:26.413134 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:26.413107 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" path="/var/lib/kubelet/pods/6d1d211b-3cb0-4f1f-a34a-b271ed0f0119/volumes" Jun 11 23:03:26.689431 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:26.689397 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:26.789374 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:26.789338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:32.672771 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:32.672732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:03:44.196137 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.196106 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45"] Jun 11 23:03:44.196503 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.196304 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" podUID="7586a387-706b-4d33-9ad1-088616c7ac89" containerName="manager" containerID="cri-o://6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef" gracePeriod=10 Jun 11 23:03:44.443288 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.443259 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:44.530534 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.530446 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x92xq\" (UniqueName: \"kubernetes.io/projected/7586a387-706b-4d33-9ad1-088616c7ac89-kube-api-access-x92xq\") pod \"7586a387-706b-4d33-9ad1-088616c7ac89\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " Jun 11 23:03:44.530534 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.530506 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586a387-706b-4d33-9ad1-088616c7ac89-extensions-socket-volume\") pod \"7586a387-706b-4d33-9ad1-088616c7ac89\" (UID: \"7586a387-706b-4d33-9ad1-088616c7ac89\") " Jun 11 23:03:44.530991 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.530963 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7586a387-706b-4d33-9ad1-088616c7ac89-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7586a387-706b-4d33-9ad1-088616c7ac89" (UID: "7586a387-706b-4d33-9ad1-088616c7ac89"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 23:03:44.532555 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.532521 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7586a387-706b-4d33-9ad1-088616c7ac89-kube-api-access-x92xq" (OuterVolumeSpecName: "kube-api-access-x92xq") pod "7586a387-706b-4d33-9ad1-088616c7ac89" (UID: "7586a387-706b-4d33-9ad1-088616c7ac89"). InnerVolumeSpecName "kube-api-access-x92xq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 23:03:44.631494 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.631464 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x92xq\" (UniqueName: \"kubernetes.io/projected/7586a387-706b-4d33-9ad1-088616c7ac89-kube-api-access-x92xq\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:44.631494 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:44.631489 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586a387-706b-4d33-9ad1-088616c7ac89-extensions-socket-volume\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:03:45.043497 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.043463 2562 generic.go:356] "Generic (PLEG): container finished" podID="7586a387-706b-4d33-9ad1-088616c7ac89" containerID="6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef" exitCode=0 Jun 11 23:03:45.043690 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.043506 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" Jun 11 23:03:45.043690 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.043542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" event={"ID":"7586a387-706b-4d33-9ad1-088616c7ac89","Type":"ContainerDied","Data":"6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef"} Jun 11 23:03:45.043690 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.043592 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45" event={"ID":"7586a387-706b-4d33-9ad1-088616c7ac89","Type":"ContainerDied","Data":"b0fb741baa9f656eddd9fb43170dac96e1a3fd16ecd4b186f616e6a2188a6148"} Jun 11 23:03:45.043690 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.043620 2562 scope.go:117] "RemoveContainer" containerID="6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef" Jun 11 23:03:45.052717 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.052695 2562 scope.go:117] "RemoveContainer" containerID="6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef" Jun 11 23:03:45.052994 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:03:45.052974 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef\": container with ID starting with 6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef not found: ID does not exist" containerID="6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef" Jun 11 23:03:45.053043 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.053003 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef"} err="failed to get container status \"6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef\": rpc error: code = NotFound desc = could not find container \"6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef\": container with ID starting with 6118c5445b504aa1ffa3587b97238575305b19425d41c8842692a3f7b19269ef not found: ID does not exist" Jun 11 23:03:45.068475 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.068449 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45"] Jun 11 23:03:45.073526 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:45.073505 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95s45"] Jun 11 23:03:46.412754 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:03:46.412724 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7586a387-706b-4d33-9ad1-088616c7ac89" path="/var/lib/kubelet/pods/7586a387-706b-4d33-9ad1-088616c7ac89/volumes" Jun 11 23:04:00.581154 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:00.581094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:00.866447 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:00.866356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:02.915612 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.915582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:02.923200 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923176 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:02.923508 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923496 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" containerName="console" Jun 11 23:04:02.923550 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923510 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" containerName="console" Jun 11 23:04:02.923550 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923534 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7586a387-706b-4d33-9ad1-088616c7ac89" containerName="manager" Jun 11 23:04:02.923550 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923541 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7586a387-706b-4d33-9ad1-088616c7ac89" containerName="manager" Jun 11 23:04:02.923645 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923589 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7586a387-706b-4d33-9ad1-088616c7ac89" containerName="manager" Jun 11 23:04:02.923645 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.923598 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d1d211b-3cb0-4f1f-a34a-b271ed0f0119" containerName="console" Jun 11 23:04:02.926576 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.926558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:02.929327 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.929297 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jzsps\"" Jun 11 23:04:02.929466 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.929442 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 11 23:04:02.937081 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.937059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:02.991254 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.991221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c1366f-692c-40ee-912b-287db3b75ddb-config-file\") pod \"limitador-limitador-64c94fd64f-v5hzr\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:02.991447 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:02.991311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fxzg\" (UniqueName: \"kubernetes.io/projected/79c1366f-692c-40ee-912b-287db3b75ddb-kube-api-access-9fxzg\") pod \"limitador-limitador-64c94fd64f-v5hzr\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:03.022561 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.022531 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:03.033119 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.033086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:03.092078 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.092050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c1366f-692c-40ee-912b-287db3b75ddb-config-file\") pod \"limitador-limitador-64c94fd64f-v5hzr\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:03.092262 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.092101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fxzg\" (UniqueName: \"kubernetes.io/projected/79c1366f-692c-40ee-912b-287db3b75ddb-kube-api-access-9fxzg\") pod \"limitador-limitador-64c94fd64f-v5hzr\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:03.092704 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.092681 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c1366f-692c-40ee-912b-287db3b75ddb-config-file\") pod \"limitador-limitador-64c94fd64f-v5hzr\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:03.096075 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.096043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:03.101263 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.101245 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fxzg\" (UniqueName: \"kubernetes.io/projected/79c1366f-692c-40ee-912b-287db3b75ddb-kube-api-access-9fxzg\") pod \"limitador-limitador-64c94fd64f-v5hzr\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:03.236581 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.236542 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:03.369273 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.369241 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:03.372626 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:04:03.372602 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c1366f_692c_40ee_912b_287db3b75ddb.slice/crio-a2d61f109a728baffd9eb2a4bfc616e3ae1f4c4dc73fe7530fa6cd4477cdafca WatchSource:0}: Error finding container a2d61f109a728baffd9eb2a4bfc616e3ae1f4c4dc73fe7530fa6cd4477cdafca: Status 404 returned error can't find the container with id a2d61f109a728baffd9eb2a4bfc616e3ae1f4c4dc73fe7530fa6cd4477cdafca Jun 11 23:04:03.813516 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.813476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:03.914320 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:03.914282 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:04.142916 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:04.142811 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" event={"ID":"79c1366f-692c-40ee-912b-287db3b75ddb","Type":"ContainerStarted","Data":"a2d61f109a728baffd9eb2a4bfc616e3ae1f4c4dc73fe7530fa6cd4477cdafca"} Jun 11 23:04:06.155357 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:06.155321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" event={"ID":"79c1366f-692c-40ee-912b-287db3b75ddb","Type":"ContainerStarted","Data":"70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a"} Jun 11 23:04:06.155791 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:06.155434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:06.176718 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:06.176671 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" podStartSLOduration=1.541193349 podStartE2EDuration="4.176656346s" totalImagesPullingTime="2.635462997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:04:06.174155938 +0000 UTC m=+698.119404848" watchObservedRunningTime="2026-06-11 23:04:06.176656346 +0000 UTC m=+698.121905237" Jun 11 23:04:17.161558 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:17.161522 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:20.021188 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:20.021151 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:20.021590 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:20.021399 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" podUID="79c1366f-692c-40ee-912b-287db3b75ddb" containerName="limitador" containerID="cri-o://70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a" gracePeriod=30 Jun 11 23:04:20.970253 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:20.970226 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:21.043368 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.043334 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fxzg\" (UniqueName: \"kubernetes.io/projected/79c1366f-692c-40ee-912b-287db3b75ddb-kube-api-access-9fxzg\") pod \"79c1366f-692c-40ee-912b-287db3b75ddb\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " Jun 11 23:04:21.043744 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.043412 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c1366f-692c-40ee-912b-287db3b75ddb-config-file\") pod \"79c1366f-692c-40ee-912b-287db3b75ddb\" (UID: \"79c1366f-692c-40ee-912b-287db3b75ddb\") " Jun 11 23:04:21.043744 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.043732 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c1366f-692c-40ee-912b-287db3b75ddb-config-file" (OuterVolumeSpecName: "config-file") pod "79c1366f-692c-40ee-912b-287db3b75ddb" (UID: "79c1366f-692c-40ee-912b-287db3b75ddb"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 23:04:21.045422 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.045395 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c1366f-692c-40ee-912b-287db3b75ddb-kube-api-access-9fxzg" (OuterVolumeSpecName: "kube-api-access-9fxzg") pod "79c1366f-692c-40ee-912b-287db3b75ddb" (UID: "79c1366f-692c-40ee-912b-287db3b75ddb"). InnerVolumeSpecName "kube-api-access-9fxzg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 23:04:21.143966 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.143871 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fxzg\" (UniqueName: \"kubernetes.io/projected/79c1366f-692c-40ee-912b-287db3b75ddb-kube-api-access-9fxzg\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:04:21.143966 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.143903 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c1366f-692c-40ee-912b-287db3b75ddb-config-file\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:04:21.234590 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.234556 2562 generic.go:356] "Generic (PLEG): container finished" podID="79c1366f-692c-40ee-912b-287db3b75ddb" containerID="70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a" exitCode=0 Jun 11 23:04:21.234765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.234599 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" Jun 11 23:04:21.234765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.234634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" event={"ID":"79c1366f-692c-40ee-912b-287db3b75ddb","Type":"ContainerDied","Data":"70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a"} Jun 11 23:04:21.234765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.234681 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-v5hzr" event={"ID":"79c1366f-692c-40ee-912b-287db3b75ddb","Type":"ContainerDied","Data":"a2d61f109a728baffd9eb2a4bfc616e3ae1f4c4dc73fe7530fa6cd4477cdafca"} Jun 11 23:04:21.234765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.234704 2562 scope.go:117] "RemoveContainer" containerID="70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a" Jun 11 23:04:21.243354 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.243339 2562 scope.go:117] "RemoveContainer" containerID="70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a" Jun 11 23:04:21.243654 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:04:21.243635 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a\": container with ID starting with 70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a not found: ID does not exist" containerID="70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a" Jun 11 23:04:21.243715 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.243662 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a"} err="failed to get container status \"70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a\": rpc error: code = NotFound desc = could not find container \"70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a\": container with ID starting with 70aea1eb6dcd6f8a0cccf22c8ec0e3ffc32d58eb32a7d02212d61c339875e58a not found: ID does not exist" Jun 11 23:04:21.255634 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.255605 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:21.259367 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:21.259346 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-v5hzr"] Jun 11 23:04:22.413107 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:22.413073 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c1366f-692c-40ee-912b-287db3b75ddb" path="/var/lib/kubelet/pods/79c1366f-692c-40ee-912b-287db3b75ddb/volumes" Jun 11 23:04:24.479289 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.479257 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-mg8pr"] Jun 11 23:04:24.479768 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.479752 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c1366f-692c-40ee-912b-287db3b75ddb" containerName="limitador" Jun 11 23:04:24.479814 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.479771 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c1366f-692c-40ee-912b-287db3b75ddb" containerName="limitador" Jun 11 23:04:24.479892 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.479881 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c1366f-692c-40ee-912b-287db3b75ddb" containerName="limitador" Jun 11 23:04:24.482782 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.482760 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.485373 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.485355 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 11 23:04:24.489663 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.489635 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mg8pr"] Jun 11 23:04:24.513532 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.513481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:24.571625 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.571579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shwqj\" (UniqueName: \"kubernetes.io/projected/d2babdef-8877-472a-b01c-a8449aaac9e0-kube-api-access-shwqj\") pod \"postgres-69fb4594cf-mg8pr\" (UID: \"d2babdef-8877-472a-b01c-a8449aaac9e0\") " pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.571861 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.571739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d2babdef-8877-472a-b01c-a8449aaac9e0-data\") pod \"postgres-69fb4594cf-mg8pr\" (UID: \"d2babdef-8877-472a-b01c-a8449aaac9e0\") " pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.672431 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.672397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d2babdef-8877-472a-b01c-a8449aaac9e0-data\") pod \"postgres-69fb4594cf-mg8pr\" (UID: \"d2babdef-8877-472a-b01c-a8449aaac9e0\") " pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.672590 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.672448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shwqj\" (UniqueName: \"kubernetes.io/projected/d2babdef-8877-472a-b01c-a8449aaac9e0-kube-api-access-shwqj\") pod \"postgres-69fb4594cf-mg8pr\" (UID: \"d2babdef-8877-472a-b01c-a8449aaac9e0\") " pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.672768 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.672749 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d2babdef-8877-472a-b01c-a8449aaac9e0-data\") pod \"postgres-69fb4594cf-mg8pr\" (UID: \"d2babdef-8877-472a-b01c-a8449aaac9e0\") " pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.681867 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.681813 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shwqj\" (UniqueName: \"kubernetes.io/projected/d2babdef-8877-472a-b01c-a8449aaac9e0-kube-api-access-shwqj\") pod \"postgres-69fb4594cf-mg8pr\" (UID: \"d2babdef-8877-472a-b01c-a8449aaac9e0\") " pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.795036 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.794955 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:24.920267 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:24.920232 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mg8pr"] Jun 11 23:04:24.923886 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:04:24.923856 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2babdef_8877_472a_b01c_a8449aaac9e0.slice/crio-3f5cdba9985cb599e0709bb1c44cfbc91e10af2b1dbf0889f30f6bc555d1191f WatchSource:0}: Error finding container 3f5cdba9985cb599e0709bb1c44cfbc91e10af2b1dbf0889f30f6bc555d1191f: Status 404 returned error can't find the container with id 3f5cdba9985cb599e0709bb1c44cfbc91e10af2b1dbf0889f30f6bc555d1191f Jun 11 23:04:25.254751 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:25.254723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mg8pr" event={"ID":"d2babdef-8877-472a-b01c-a8449aaac9e0","Type":"ContainerStarted","Data":"3f5cdba9985cb599e0709bb1c44cfbc91e10af2b1dbf0889f30f6bc555d1191f"} Jun 11 23:04:28.272589 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:28.272542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mg8pr" event={"ID":"d2babdef-8877-472a-b01c-a8449aaac9e0","Type":"ContainerStarted","Data":"a956336ad0afeedd1c4d74ca2fa300f87202ecaf636cac077ab72d432192f691"} Jun 11 23:04:28.273062 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:28.272726 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:28.287835 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:28.287793 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-mg8pr" podStartSLOduration=1.923807126 podStartE2EDuration="4.287779163s" totalImagesPullingTime="2.363972037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:04:28.286968414 +0000 UTC m=+720.232217305" watchObservedRunningTime="2026-06-11 23:04:28.287779163 +0000 UTC m=+720.233028060" Jun 11 23:04:34.314633 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:34.314607 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-mg8pr" Jun 11 23:04:34.650218 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:34.650125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:34.738977 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:34.738940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:34.864036 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:34.864001 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:04:44.210859 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.210817 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-vxxpm"] Jun 11 23:04:44.220384 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.220354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" Jun 11 23:04:44.225671 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.225645 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 23:04:44.226184 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.226167 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 23:04:44.227000 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.226980 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-phdhb\"" Jun 11 23:04:44.231859 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.231818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-vxxpm"] Jun 11 23:04:44.341240 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.341210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4qv9\" (UniqueName: \"kubernetes.io/projected/6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2-kube-api-access-w4qv9\") pod \"keycloak-operator-d67dbbb48-vxxpm\" (UID: \"6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2\") " pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" Jun 11 23:04:44.441901 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.441862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4qv9\" (UniqueName: \"kubernetes.io/projected/6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2-kube-api-access-w4qv9\") pod \"keycloak-operator-d67dbbb48-vxxpm\" (UID: \"6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2\") " pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" Jun 11 23:04:44.451978 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.451949 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4qv9\" (UniqueName: \"kubernetes.io/projected/6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2-kube-api-access-w4qv9\") pod \"keycloak-operator-d67dbbb48-vxxpm\" (UID: \"6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2\") " pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" Jun 11 23:04:44.530735 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.530659 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" Jun 11 23:04:44.683422 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:44.683391 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-vxxpm"] Jun 11 23:04:44.686735 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:04:44.686706 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3d2bb0_5dcd_4a2a_8cff_819c62b7a9e2.slice/crio-8496c5e3149cb70b4946ea53f55b72a498b2462f1ba05d4f3e357eecb8fcad88 WatchSource:0}: Error finding container 8496c5e3149cb70b4946ea53f55b72a498b2462f1ba05d4f3e357eecb8fcad88: Status 404 returned error can't find the container with id 8496c5e3149cb70b4946ea53f55b72a498b2462f1ba05d4f3e357eecb8fcad88 Jun 11 23:04:45.353398 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:45.353323 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" event={"ID":"6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2","Type":"ContainerStarted","Data":"8496c5e3149cb70b4946ea53f55b72a498b2462f1ba05d4f3e357eecb8fcad88"} Jun 11 23:04:52.391392 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:52.391350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" event={"ID":"6d3d2bb0-5dcd-4a2a-8cff-819c62b7a9e2","Type":"ContainerStarted","Data":"09eec113a23cb4e68eb559affe970535be73608003698dac567873ba1485c4bf"} Jun 11 23:04:52.413506 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:04:52.413458 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-vxxpm" podStartSLOduration=1.4913624429999999 podStartE2EDuration="8.413442223s" totalImagesPullingTime="6.92207978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:04:52.411783912 +0000 UTC m=+744.357032817" watchObservedRunningTime="2026-06-11 23:04:52.413442223 +0000 UTC m=+744.358691114" Jun 11 23:05:00.458750 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:00.458708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:00.560432 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:00.560391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:31.956589 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:31.956547 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:32.292480 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:32.292395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:38.238510 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.238475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:38.253717 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.253689 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c76c569d4-r9tqc"] Jun 11 23:05:38.260264 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.260226 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:38.263027 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.263008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 23:05:38.263118 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.263050 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cwww4\"" Jun 11 23:05:38.263118 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.263010 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 23:05:38.269719 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.269692 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c76c569d4-r9tqc"] Jun 11 23:05:38.341465 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.341416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:38.410675 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.410644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78vjq\" (UniqueName: \"kubernetes.io/projected/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-kube-api-access-78vjq\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:38.410904 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.410802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:38.435065 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.435031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:38.439325 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.439300 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85b78b4694-x727z"] Jun 11 23:05:38.441579 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.441562 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:38.453427 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.453409 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85b78b4694-x727z"] Jun 11 23:05:38.511585 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.511496 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drgj6\" (UniqueName: \"kubernetes.io/projected/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-kube-api-access-drgj6\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:38.511740 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.511588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78vjq\" (UniqueName: \"kubernetes.io/projected/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-kube-api-access-78vjq\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:38.511740 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.511659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-webhook-cert\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:38.511740 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.511715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:38.511884 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:05:38.511822 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 11 23:05:38.511924 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:05:38.511892 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert podName:bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0 nodeName:}" failed. No retries permitted until 2026-06-11 23:05:39.011876918 +0000 UTC m=+790.957125786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert") pod "maas-controller-6c76c569d4-r9tqc" (UID: "bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0") : secret "maas-controller-webhook-cert" not found Jun 11 23:05:38.520552 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.520516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78vjq\" (UniqueName: \"kubernetes.io/projected/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-kube-api-access-78vjq\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:38.612129 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.612102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-webhook-cert\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:38.612314 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:05:38.612189 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 11 23:05:38.612314 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.612192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drgj6\" (UniqueName: \"kubernetes.io/projected/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-kube-api-access-drgj6\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:38.612314 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:05:38.612288 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-webhook-cert podName:97cab4ae-d4e1-4021-85a5-3953e2c5ab35 nodeName:}" failed. No retries permitted until 2026-06-11 23:05:39.112267617 +0000 UTC m=+791.057516487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-webhook-cert") pod "maas-controller-85b78b4694-x727z" (UID: "97cab4ae-d4e1-4021-85a5-3953e2c5ab35") : secret "maas-controller-webhook-cert" not found Jun 11 23:05:38.625123 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.625100 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drgj6\" (UniqueName: \"kubernetes.io/projected/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-kube-api-access-drgj6\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:38.836222 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:38.836122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:39.014927 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.014877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:39.017346 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.017323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert\") pod \"maas-controller-6c76c569d4-r9tqc\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:39.115941 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.115824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-webhook-cert\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:39.118252 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.118231 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97cab4ae-d4e1-4021-85a5-3953e2c5ab35-webhook-cert\") pod \"maas-controller-85b78b4694-x727z\" (UID: \"97cab4ae-d4e1-4021-85a5-3953e2c5ab35\") " pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:39.170478 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.170445 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:39.306090 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.306066 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c76c569d4-r9tqc"] Jun 11 23:05:39.308441 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:05:39.308415 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1fb9f1_2781_47c0_ad18_b0285a6c5fe0.slice/crio-7af1b1d8fd36023094d61aa0fd9deeb398638d885396ae7b8ee19eba317acaa2 WatchSource:0}: Error finding container 7af1b1d8fd36023094d61aa0fd9deeb398638d885396ae7b8ee19eba317acaa2: Status 404 returned error can't find the container with id 7af1b1d8fd36023094d61aa0fd9deeb398638d885396ae7b8ee19eba317acaa2 Jun 11 23:05:39.309678 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.309663 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 23:05:39.352006 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.351973 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:39.481081 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.481049 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85b78b4694-x727z"] Jun 11 23:05:39.483954 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:05:39.483922 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97cab4ae_d4e1_4021_85a5_3953e2c5ab35.slice/crio-1b77bd4b9552bdfb02b6d3d8eca6781244d3162dc45fead927075d0b726f326d WatchSource:0}: Error finding container 1b77bd4b9552bdfb02b6d3d8eca6781244d3162dc45fead927075d0b726f326d: Status 404 returned error can't find the container with id 1b77bd4b9552bdfb02b6d3d8eca6781244d3162dc45fead927075d0b726f326d Jun 11 23:05:39.630717 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.630684 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" event={"ID":"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0","Type":"ContainerStarted","Data":"7af1b1d8fd36023094d61aa0fd9deeb398638d885396ae7b8ee19eba317acaa2"} Jun 11 23:05:39.631618 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:39.631594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85b78b4694-x727z" event={"ID":"97cab4ae-d4e1-4021-85a5-3953e2c5ab35","Type":"ContainerStarted","Data":"1b77bd4b9552bdfb02b6d3d8eca6781244d3162dc45fead927075d0b726f326d"} Jun 11 23:05:41.415230 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.415128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:41.516064 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.516018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:41.643446 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.643414 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85b78b4694-x727z" event={"ID":"97cab4ae-d4e1-4021-85a5-3953e2c5ab35","Type":"ContainerStarted","Data":"0e0dcd6459930c06cb8458ad6dc72026e433d7a874b8549f33cc6b47fa1fab36"} Jun 11 23:05:41.643633 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.643496 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:41.644982 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.644955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" event={"ID":"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0","Type":"ContainerStarted","Data":"5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e"} Jun 11 23:05:41.645108 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.645094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:41.675972 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.675867 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85b78b4694-x727z" podStartSLOduration=1.9882282359999999 podStartE2EDuration="3.675834893s" totalImagesPullingTime="1.687606657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:05:41.674435682 +0000 UTC m=+793.619684598" watchObservedRunningTime="2026-06-11 23:05:41.675834893 +0000 UTC m=+793.621083781" Jun 11 23:05:41.710726 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:41.710681 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" podStartSLOduration=1.844142663 podStartE2EDuration="3.710665667s" totalImagesPullingTime="1.866523004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:05:41.707907416 +0000 UTC m=+793.653156328" watchObservedRunningTime="2026-06-11 23:05:41.710665667 +0000 UTC m=+793.655914556" Jun 11 23:05:43.469475 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:43.469437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:05:52.655193 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:52.655113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:52.655193 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:52.655184 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85b78b4694-x727z" Jun 11 23:05:52.726117 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:52.726089 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c76c569d4-r9tqc"] Jun 11 23:05:52.726306 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:52.726280 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" podUID="bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" containerName="manager" containerID="cri-o://5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e" gracePeriod=30 Jun 11 23:05:52.972915 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:52.972887 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:53.143038 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.143005 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert\") pod \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " Jun 11 23:05:53.143219 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.143095 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78vjq\" (UniqueName: \"kubernetes.io/projected/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-kube-api-access-78vjq\") pod \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\" (UID: \"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0\") " Jun 11 23:05:53.145261 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.145233 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" (UID: "bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 23:05:53.145375 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.145269 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-kube-api-access-78vjq" (OuterVolumeSpecName: "kube-api-access-78vjq") pod "bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" (UID: "bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0"). InnerVolumeSpecName "kube-api-access-78vjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 23:05:53.244589 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.244511 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-webhook-cert\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:05:53.244589 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.244540 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78vjq\" (UniqueName: \"kubernetes.io/projected/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0-kube-api-access-78vjq\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:05:53.704400 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.704370 2562 generic.go:356] "Generic (PLEG): container finished" podID="bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" containerID="5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e" exitCode=0 Jun 11 23:05:53.704765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.704417 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" Jun 11 23:05:53.704765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.704452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" event={"ID":"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0","Type":"ContainerDied","Data":"5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e"} Jun 11 23:05:53.704765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.704502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c76c569d4-r9tqc" event={"ID":"bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0","Type":"ContainerDied","Data":"7af1b1d8fd36023094d61aa0fd9deeb398638d885396ae7b8ee19eba317acaa2"} Jun 11 23:05:53.704765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.704523 2562 scope.go:117] "RemoveContainer" containerID="5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e" Jun 11 23:05:53.713626 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.713612 2562 scope.go:117] "RemoveContainer" containerID="5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e" Jun 11 23:05:53.713911 ip-10-0-138-236 kubenswrapper[2562]: E0611 23:05:53.713887 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e\": container with ID starting with 5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e not found: ID does not exist" containerID="5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e" Jun 11 23:05:53.713974 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.713919 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e"} err="failed to get container status \"5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e\": rpc error: code = NotFound desc = could not find container \"5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e\": container with ID starting with 5caf62297cd201b99663fc838dbdf5998082b3f6d0abe8af21647e1602f5983e not found: ID does not exist" Jun 11 23:05:53.726668 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.726647 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c76c569d4-r9tqc"] Jun 11 23:05:53.729481 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:53.729459 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c76c569d4-r9tqc"] Jun 11 23:05:54.413075 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:05:54.413047 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" path="/var/lib/kubelet/pods/bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0/volumes" Jun 11 23:06:10.559939 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.559903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:06:10.677714 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.677680 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-59dcb9b5b7-cxb2j"] Jun 11 23:06:10.678181 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.678165 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" containerName="manager" Jun 11 23:06:10.678232 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.678183 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" containerName="manager" Jun 11 23:06:10.678267 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.678245 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1fb9f1-2781-47c0-ad18-b0285a6c5fe0" containerName="manager" Jun 11 23:06:10.682695 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.682679 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.685860 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.685826 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 11 23:06:10.685970 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.685886 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-ch4s4\"" Jun 11 23:06:10.694630 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.694604 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-59dcb9b5b7-cxb2j"] Jun 11 23:06:10.782765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.782733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64aabc1-d855-4f8f-9824-dd5987e55812-maas-api-tls\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.782765 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.782767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcsc6\" (UniqueName: \"kubernetes.io/projected/d64aabc1-d855-4f8f-9824-dd5987e55812-kube-api-access-tcsc6\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.783027 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.782798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64aabc1-d855-4f8f-9824-dd5987e55812-openshift-service-ca\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.883816 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.883734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64aabc1-d855-4f8f-9824-dd5987e55812-maas-api-tls\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.883816 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.883783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcsc6\" (UniqueName: \"kubernetes.io/projected/d64aabc1-d855-4f8f-9824-dd5987e55812-kube-api-access-tcsc6\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.884011 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.883830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64aabc1-d855-4f8f-9824-dd5987e55812-openshift-service-ca\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.884487 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.884466 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64aabc1-d855-4f8f-9824-dd5987e55812-openshift-service-ca\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.886198 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.886178 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64aabc1-d855-4f8f-9824-dd5987e55812-maas-api-tls\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.905976 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.905941 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcsc6\" (UniqueName: \"kubernetes.io/projected/d64aabc1-d855-4f8f-9824-dd5987e55812-kube-api-access-tcsc6\") pod \"maas-api-59dcb9b5b7-cxb2j\" (UID: \"d64aabc1-d855-4f8f-9824-dd5987e55812\") " pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:10.994159 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:10.994118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:11.135272 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.135244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-59dcb9b5b7-cxb2j"] Jun 11 23:06:11.138565 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:06:11.138539 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64aabc1_d855_4f8f_9824_dd5987e55812.slice/crio-e70ea8b0a2f0a953aa98ea9cb69147b48882bccef618bdebbf2a4114cc12b881 WatchSource:0}: Error finding container e70ea8b0a2f0a953aa98ea9cb69147b48882bccef618bdebbf2a4114cc12b881: Status 404 returned error can't find the container with id e70ea8b0a2f0a953aa98ea9cb69147b48882bccef618bdebbf2a4114cc12b881 Jun 11 23:06:11.221077 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.221042 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-j7567"] Jun 11 23:06:11.225554 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.225534 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:06:11.228272 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.228252 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 23:06:11.228272 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.228265 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-x4pwh\"" Jun 11 23:06:11.234786 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.234762 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-j7567"] Jun 11 23:06:11.255502 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.255476 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-x78k4"] Jun 11 23:06:11.258763 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.258748 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:11.271121 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.271096 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-x78k4"] Jun 11 23:06:11.288161 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.288134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w68w6\" (UniqueName: \"kubernetes.io/projected/eae48b67-0f65-4019-85bc-b2e4f0b80dd1-kube-api-access-w68w6\") pod \"payload-processing-699b6cd6c-x78k4\" (UID: \"eae48b67-0f65-4019-85bc-b2e4f0b80dd1\") " pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:11.288300 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.288176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqxtl\" (UniqueName: \"kubernetes.io/projected/b28e4b34-231c-4707-9c2f-1ae8cff76bfe-kube-api-access-lqxtl\") pod \"payload-pre-processing-579785bc64-j7567\" (UID: \"b28e4b34-231c-4707-9c2f-1ae8cff76bfe\") " pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:06:11.389313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.389228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w68w6\" (UniqueName: \"kubernetes.io/projected/eae48b67-0f65-4019-85bc-b2e4f0b80dd1-kube-api-access-w68w6\") pod \"payload-processing-699b6cd6c-x78k4\" (UID: \"eae48b67-0f65-4019-85bc-b2e4f0b80dd1\") " pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:11.389313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.389269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqxtl\" (UniqueName: \"kubernetes.io/projected/b28e4b34-231c-4707-9c2f-1ae8cff76bfe-kube-api-access-lqxtl\") pod \"payload-pre-processing-579785bc64-j7567\" (UID: \"b28e4b34-231c-4707-9c2f-1ae8cff76bfe\") " pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:06:11.398157 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.398126 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqxtl\" (UniqueName: \"kubernetes.io/projected/b28e4b34-231c-4707-9c2f-1ae8cff76bfe-kube-api-access-lqxtl\") pod \"payload-pre-processing-579785bc64-j7567\" (UID: \"b28e4b34-231c-4707-9c2f-1ae8cff76bfe\") " pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:06:11.398294 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.398162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w68w6\" (UniqueName: \"kubernetes.io/projected/eae48b67-0f65-4019-85bc-b2e4f0b80dd1-kube-api-access-w68w6\") pod \"payload-processing-699b6cd6c-x78k4\" (UID: \"eae48b67-0f65-4019-85bc-b2e4f0b80dd1\") " pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:11.537033 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.536993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:06:11.568055 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.568027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:11.642699 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.642523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:06:11.694746 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.694719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-j7567"] Jun 11 23:06:11.696564 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:06:11.696528 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28e4b34_231c_4707_9c2f_1ae8cff76bfe.slice/crio-81a29b4e9ec41c30a1e8d9e68aa5f90a6d276d48a1b0d599756e6d14bcab1ad6 WatchSource:0}: Error finding container 81a29b4e9ec41c30a1e8d9e68aa5f90a6d276d48a1b0d599756e6d14bcab1ad6: Status 404 returned error can't find the container with id 81a29b4e9ec41c30a1e8d9e68aa5f90a6d276d48a1b0d599756e6d14bcab1ad6 Jun 11 23:06:11.730697 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.730669 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-x78k4"] Jun 11 23:06:11.732045 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:06:11.732006 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae48b67_0f65_4019_85bc_b2e4f0b80dd1.slice/crio-13b3b5bfd05890c97c08b315c38c4bfed74ca868b047e8fef52b36c045ea8dee WatchSource:0}: Error finding container 13b3b5bfd05890c97c08b315c38c4bfed74ca868b047e8fef52b36c045ea8dee: Status 404 returned error can't find the container with id 13b3b5bfd05890c97c08b315c38c4bfed74ca868b047e8fef52b36c045ea8dee Jun 11 23:06:11.795658 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.795622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" event={"ID":"d64aabc1-d855-4f8f-9824-dd5987e55812","Type":"ContainerStarted","Data":"e70ea8b0a2f0a953aa98ea9cb69147b48882bccef618bdebbf2a4114cc12b881"} Jun 11 23:06:11.797186 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.797152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" event={"ID":"b28e4b34-231c-4707-9c2f-1ae8cff76bfe","Type":"ContainerStarted","Data":"81a29b4e9ec41c30a1e8d9e68aa5f90a6d276d48a1b0d599756e6d14bcab1ad6"} Jun 11 23:06:11.798466 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:11.798442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" event={"ID":"eae48b67-0f65-4019-85bc-b2e4f0b80dd1","Type":"ContainerStarted","Data":"13b3b5bfd05890c97c08b315c38c4bfed74ca868b047e8fef52b36c045ea8dee"} Jun 11 23:06:13.813593 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:13.813544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" event={"ID":"d64aabc1-d855-4f8f-9824-dd5987e55812","Type":"ContainerStarted","Data":"97ac46a420ae1d23bbb8c7eac588907f4fb8afd60f1b98842a2b8aaf342d1a74"} Jun 11 23:06:13.814062 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:13.813624 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:13.832346 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:13.832288 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" podStartSLOduration=1.808240542 podStartE2EDuration="3.832266364s" totalImagesPullingTime="2.024025822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:06:13.829890472 +0000 UTC m=+825.775139364" watchObservedRunningTime="2026-06-11 23:06:13.832266364 +0000 UTC m=+825.777515257" Jun 11 23:06:14.820739 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:14.820699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" event={"ID":"b28e4b34-231c-4707-9c2f-1ae8cff76bfe","Type":"ContainerStarted","Data":"660385d3cbdbbab5d0bd7b5846f5deae1f4e39318b54a023746c1b1c9dda42ce"} Jun 11 23:06:14.821242 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:14.820869 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:06:14.822329 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:14.822296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" event={"ID":"eae48b67-0f65-4019-85bc-b2e4f0b80dd1","Type":"ContainerStarted","Data":"873eabfed19c470dd339702b66adc2c49d954b9297a6049fea8c76df947eae7c"} Jun 11 23:06:14.822444 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:14.822359 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:14.840118 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:14.840073 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" podStartSLOduration=1.2728363360000001 podStartE2EDuration="3.840059246s" totalImagesPullingTime="2.56722291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:06:14.837993442 +0000 UTC m=+826.783242353" watchObservedRunningTime="2026-06-11 23:06:14.840059246 +0000 UTC m=+826.785308128" Jun 11 23:06:14.866209 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:14.866154 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" podStartSLOduration=1.341927674 podStartE2EDuration="3.866137767s" totalImagesPullingTime="2.524210093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:06:14.864317746 +0000 UTC m=+826.809566637" watchObservedRunningTime="2026-06-11 23:06:14.866137767 +0000 UTC m=+826.811386654" Jun 11 23:06:19.828276 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:19.828247 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-59dcb9b5b7-cxb2j" Jun 11 23:06:25.829293 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:25.829262 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-x78k4" Jun 11 23:06:25.829748 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:06:25.829371 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-j7567" Jun 11 23:07:10.732261 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:10.732228 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:10.832327 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:10.832292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:15.871790 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:15.871754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:25.652632 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.651522 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm"] Jun 11 23:07:25.655728 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.655705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.658891 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.658834 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-l74jk\"" Jun 11 23:07:25.658891 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.658863 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 23:07:25.659954 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.659923 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 23:07:25.660080 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.659999 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 11 23:07:25.668376 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.668350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm"] Jun 11 23:07:25.691779 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.691746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:25.750859 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.750819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.751015 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.750903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.751015 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.750928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct4k7\" (UniqueName: \"kubernetes.io/projected/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-kube-api-access-ct4k7\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.751015 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.750949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.751121 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.751020 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.751121 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.751045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.851987 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.851951 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.851987 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.851990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4k7\" (UniqueName: \"kubernetes.io/projected/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-kube-api-access-ct4k7\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852075 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852520 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852469 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852520 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852484 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.852590 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.852575 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.854411 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.854390 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.854566 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.854549 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.861308 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.861279 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4k7\" (UniqueName: \"kubernetes.io/projected/97f7072a-bec3-4125-ac8e-2fb8d99ca0fc-kube-api-access-ct4k7\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ls6xm\" (UID: \"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:25.989028 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:25.988984 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:26.152214 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:26.152175 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm"] Jun 11 23:07:26.155375 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:07:26.155338 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f7072a_bec3_4125_ac8e_2fb8d99ca0fc.slice/crio-264fead88f5e0ed26b80a00d8fd72995720ed55ac89903d49a03a44254c82b80 WatchSource:0}: Error finding container 264fead88f5e0ed26b80a00d8fd72995720ed55ac89903d49a03a44254c82b80: Status 404 returned error can't find the container with id 264fead88f5e0ed26b80a00d8fd72995720ed55ac89903d49a03a44254c82b80 Jun 11 23:07:26.165152 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:26.165121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" event={"ID":"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc","Type":"ContainerStarted","Data":"264fead88f5e0ed26b80a00d8fd72995720ed55ac89903d49a03a44254c82b80"} Jun 11 23:07:29.022541 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:29.022512 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 23:07:29.022990 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:29.022976 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 23:07:29.184953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:29.184893 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" event={"ID":"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc","Type":"ContainerStarted","Data":"28dbf244253149d827ec6e7a6ff18d3e384daafe0e2b728c48e66c8ac8a65873"} Jun 11 23:07:30.652904 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:30.652811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:30.753865 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:30.753801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:31.198203 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:31.198173 2562 generic.go:356] "Generic (PLEG): container finished" podID="97f7072a-bec3-4125-ac8e-2fb8d99ca0fc" containerID="28dbf244253149d827ec6e7a6ff18d3e384daafe0e2b728c48e66c8ac8a65873" exitCode=0 Jun 11 23:07:31.198388 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:31.198259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" event={"ID":"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc","Type":"ContainerDied","Data":"28dbf244253149d827ec6e7a6ff18d3e384daafe0e2b728c48e66c8ac8a65873"} Jun 11 23:07:33.210645 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:33.210611 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" event={"ID":"97f7072a-bec3-4125-ac8e-2fb8d99ca0fc","Type":"ContainerStarted","Data":"677fabe338ae2068c21079f0ead0c9de8a374c93cadc014b63ebdd5f841ffb21"} Jun 11 23:07:33.234476 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:33.234421 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" podStartSLOduration=3.038079176 podStartE2EDuration="8.234403054s" totalImagesPullingTime="3.915630829s" totalInitContainerRuntime="1.280693049s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:07:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:07:33.234004343 +0000 UTC m=+905.179253234" watchObservedRunningTime="2026-06-11 23:07:33.234403054 +0000 UTC m=+905.179651944" Jun 11 23:07:34.503579 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:34.503542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:35.989948 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:35.989909 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:35.990351 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:35.989977 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:36.002523 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:36.002499 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:36.237415 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:36.237388 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ls6xm" Jun 11 23:07:39.960422 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.960383 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg"] Jun 11 23:07:39.964977 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.964955 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.968245 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.968226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 11 23:07:39.984453 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.984425 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg"] Jun 11 23:07:39.985524 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.985502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.985601 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.985533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.985601 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.985552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.985601 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.985587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.985808 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.985757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4mcn\" (UniqueName: \"kubernetes.io/projected/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-kube-api-access-f4mcn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.985878 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.985818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:39.993924 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:39.993879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:40.086386 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4mcn\" (UniqueName: \"kubernetes.io/projected/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-kube-api-access-f4mcn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.086571 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.086571 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.086645 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.086695 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.086695 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.086795 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.086776 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.087046 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.087024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.087126 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.087036 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.088799 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.088777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.089089 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.089073 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.095533 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.095503 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4mcn\" (UniqueName: \"kubernetes.io/projected/5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8-kube-api-access-f4mcn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg\" (UID: \"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.275800 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.275690 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" Jun 11 23:07:40.420299 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:40.420272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg"] Jun 11 23:07:40.422235 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:07:40.422203 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c70f8e3_8156_4e1e_87cc_fce5d7c1e8c8.slice/crio-9863ae12314f768d17689e50c13457ac0339a16e7d5c46083ca0d3fabd209c73 WatchSource:0}: Error finding container 9863ae12314f768d17689e50c13457ac0339a16e7d5c46083ca0d3fabd209c73: Status 404 returned error can't find the container with id 9863ae12314f768d17689e50c13457ac0339a16e7d5c46083ca0d3fabd209c73 Jun 11 23:07:41.257740 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:41.257700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" event={"ID":"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8","Type":"ContainerStarted","Data":"e7d9c74e1a5e3b12aecfbb9df2fef064efc0f588d9cfb2700815e3cb7301175b"} Jun 11 23:07:41.258254 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:41.257755 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg" event={"ID":"5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8","Type":"ContainerStarted","Data":"9863ae12314f768d17689e50c13457ac0339a16e7d5c46083ca0d3fabd209c73"} Jun 11 23:07:48.989659 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:48.989622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:49.117813 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:49.117764 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:07:56.202788 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:07:56.202751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:09:34.536994 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:09:34.536963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:09:34.620383 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:09:34.620352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:11:15.274795 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:11:15.274761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:11:23.919148 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:11:23.919112 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:12:10.127586 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:12:10.127496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:12:10.229838 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:12:10.229807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:12:29.051311 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:12:29.051284 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 23:12:29.052956 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:12:29.052938 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzc5s_a7aaaf97-3eac-44f0-9ee0-a308e0f2e805/ovn-acl-logging/0.log" Jun 11 23:13:02.536953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:13:02.536918 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:13:02.636745 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:13:02.636707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:13:44.268675 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:13:44.268627 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:14:25.213290 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.213249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-784f85db5-xpwpm_9d75a68a-0e53-4313-b1b2-a615a90457bb/nginx/0.log" Jun 11 23:14:25.326022 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.325985 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-zfcqp_6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098/manager/0.log" Jun 11 23:14:25.440150 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.440122 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-w7hzg_aa2f432d-c676-4ef2-8053-770bebc15650/manager/0.log" Jun 11 23:14:25.555281 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.555189 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-59dcb9b5b7-cxb2j_d64aabc1-d855-4f8f-9824-dd5987e55812/maas-api/0.log" Jun 11 23:14:25.666942 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.666903 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85b78b4694-x727z_97cab4ae-d4e1-4021-85a5-3953e2c5ab35/manager/0.log" Jun 11 23:14:25.776955 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.776923 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-lxw9r_4f36dab2-ccb3-4541-99c2-f997adcf7536/server/0.log" Jun 11 23:14:25.885981 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.885891 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-mxmkn_0a853ddc-b4a3-4cdf-a408-bb138034e80d/manager/0.log" Jun 11 23:14:25.991577 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.991551 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f864d649-2hgvp_96b109cf-c44b-4a83-9130-22a6cc268ff5/copy-manifests/0.log" Jun 11 23:14:25.998962 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:25.998938 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f864d649-2hgvp_96b109cf-c44b-4a83-9130-22a6cc268ff5/manager/0.log" Jun 11 23:14:26.358192 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:26.358168 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mg8pr_d2babdef-8877-472a-b01c-a8449aaac9e0/postgres/0.log" Jun 11 23:14:27.887032 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:27.886990 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9zlcp_2d80928c-ccf1-4def-b144-d6680ffe7182/manager/0.log" Jun 11 23:14:28.982624 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:28.982597 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-bfm8m_9994305c-64de-4b66-a50b-041b96c1d7a8/discovery/0.log" Jun 11 23:14:29.094794 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:29.094771 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c495b869c-6khwv_4226b38a-6127-4aa7-8dc8-4122d4f2b076/kube-auth-proxy/0.log" Jun 11 23:14:29.423328 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:29.423249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-j7567_b28e4b34-231c-4707-9c2f-1ae8cff76bfe/payload-pre-processing/0.log" Jun 11 23:14:29.532188 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:29.532152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-x78k4_eae48b67-0f65-4019-85bc-b2e4f0b80dd1/payload-processing/0.log" Jun 11 23:14:30.098333 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:30.098291 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-ls6xm_97f7072a-bec3-4125-ac8e-2fb8d99ca0fc/main/0.log" Jun 11 23:14:30.105831 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:30.105812 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-ls6xm_97f7072a-bec3-4125-ac8e-2fb8d99ca0fc/storage-initializer/0.log" Jun 11 23:14:30.340670 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:30.340613 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cdm7xg_5c70f8e3-8156-4e1e-87cc-fce5d7c1e8c8/storage-initializer/0.log" Jun 11 23:14:35.048048 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:35.048005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:14:35.145818 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:35.145777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" limit=1024 Jun 11 23:14:37.631302 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:37.631278 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cbjj6_3d46664d-dfda-4138-aa94-ff3bb5cf5b8f/global-pull-secret-syncer/0.log" Jun 11 23:14:37.773667 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:37.773637 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ghcrg_b924723d-0016-433e-8143-dfa369f2c9d7/konnectivity-agent/0.log" Jun 11 23:14:37.883328 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:37.883260 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-236.ec2.internal_3cfdc7ebe480fabb219cc62ed7af9efa/haproxy/0.log" Jun 11 23:14:41.539711 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:41.539681 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-7577f_6085e851-2d24-4956-93ff-10a1da9d8944/sail-operator/0.log" Jun 11 23:14:42.853205 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:42.853172 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9zlcp_2d80928c-ccf1-4def-b144-d6680ffe7182/manager/0.log" Jun 11 23:14:45.130557 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.130526 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78d6c5fdf-hqdfq_49314bf2-cb4f-4880-aa1f-f05e709bc0d4/metrics-server/0.log" Jun 11 23:14:45.177051 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.177013 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-67jhk_a81f0829-d23d-4cdf-b4f0-3d71d925e7c3/monitoring-plugin/0.log" Jun 11 23:14:45.351643 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.351619 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqvxx_3ef85d25-218d-4b67-b9c7-35aedadf481c/node-exporter/0.log" Jun 11 23:14:45.380045 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.380022 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqvxx_3ef85d25-218d-4b67-b9c7-35aedadf481c/kube-rbac-proxy/0.log" Jun 11 23:14:45.407542 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.407470 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pqvxx_3ef85d25-218d-4b67-b9c7-35aedadf481c/init-textfile/0.log" Jun 11 23:14:45.881816 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.881787 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-xccvr_0d04c6c0-1605-4ac2-a33f-56e93e47abc7/prometheus-operator-admission-webhook/0.log" Jun 11 23:14:45.996075 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:45.996046 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/thanos-query/0.log" Jun 11 23:14:46.016505 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:46.016482 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy-web/0.log" Jun 11 23:14:46.036355 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:46.036316 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy/0.log" Jun 11 23:14:46.055441 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:46.055424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/prom-label-proxy/0.log" Jun 11 23:14:46.089546 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:46.089525 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy-rules/0.log" Jun 11 23:14:46.113081 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:46.113061 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b45c4cffb-7972f_17f0432b-f005-4c55-b1a4-210bba424093/kube-rbac-proxy-metrics/0.log" Jun 11 23:14:48.681396 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.681323 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-98f8597f5-9glwm_7f6663ef-71bc-487f-b572-77e8d524cdbf/console/0.log" Jun 11 23:14:48.717062 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.717036 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-hhs62_1f9a17f6-e92f-4701-ad00-2b4e1216e004/download-server/0.log" Jun 11 23:14:48.783834 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.783800 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b88td/perf-node-gather-daemonset-j248b"] Jun 11 23:14:48.787237 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.787222 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.790253 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.790227 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b88td\"/\"openshift-service-ca.crt\"" Jun 11 23:14:48.790375 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.790289 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b88td\"/\"default-dockercfg-bfjln\"" Jun 11 23:14:48.791613 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.791592 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b88td\"/\"kube-root-ca.crt\"" Jun 11 23:14:48.796129 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.796107 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b88td/perf-node-gather-daemonset-j248b"] Jun 11 23:14:48.873333 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.873297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knwb\" (UniqueName: \"kubernetes.io/projected/e7d7957d-0225-4d86-bffd-df4dae0b89b6-kube-api-access-6knwb\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.873487 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.873346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-lib-modules\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.873487 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.873397 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-podres\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.873487 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.873484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-sys\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.873587 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.873501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-proc\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974649 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-sys\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974649 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-proc\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974800 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-proc\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974800 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974704 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-sys\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974800 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knwb\" (UniqueName: \"kubernetes.io/projected/e7d7957d-0225-4d86-bffd-df4dae0b89b6-kube-api-access-6knwb\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974937 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-lib-modules\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.974937 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-podres\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.975002 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-podres\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.975002 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.974974 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7d7957d-0225-4d86-bffd-df4dae0b89b6-lib-modules\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:48.983526 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:48.983506 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knwb\" (UniqueName: \"kubernetes.io/projected/e7d7957d-0225-4d86-bffd-df4dae0b89b6-kube-api-access-6knwb\") pod \"perf-node-gather-daemonset-j248b\" (UID: \"e7d7957d-0225-4d86-bffd-df4dae0b89b6\") " pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:49.098312 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:49.098285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:49.227208 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:49.227182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b88td/perf-node-gather-daemonset-j248b"] Jun 11 23:14:49.231568 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:14:49.231533 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode7d7957d_0225_4d86_bffd_df4dae0b89b6.slice/crio-8d921f7802f7a0157a63b5b09e6b154ae81f32f01fea96fe80e966ebe433315b WatchSource:0}: Error finding container 8d921f7802f7a0157a63b5b09e6b154ae81f32f01fea96fe80e966ebe433315b: Status 404 returned error can't find the container with id 8d921f7802f7a0157a63b5b09e6b154ae81f32f01fea96fe80e966ebe433315b Jun 11 23:14:49.373839 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:49.373804 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" event={"ID":"e7d7957d-0225-4d86-bffd-df4dae0b89b6","Type":"ContainerStarted","Data":"3ecb66e77449a228b136845bc832689c7cc91dc43d08f965ddcc01760f42b1bc"} Jun 11 23:14:49.374040 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:49.373883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" event={"ID":"e7d7957d-0225-4d86-bffd-df4dae0b89b6","Type":"ContainerStarted","Data":"8d921f7802f7a0157a63b5b09e6b154ae81f32f01fea96fe80e966ebe433315b"} Jun 11 23:14:49.374040 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:49.373906 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:49.402580 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:49.402531 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" podStartSLOduration=1.402516253 podStartE2EDuration="1.402516253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 23:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:14:49.401234036 +0000 UTC m=+1341.346482947" watchObservedRunningTime="2026-06-11 23:14:49.402516253 +0000 UTC m=+1341.347765143" Jun 11 23:14:50.657953 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:50.657921 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q65s9_ba1b3401-8894-4b43-8ab1-58fe4d6ee017/dns/0.log" Jun 11 23:14:50.685443 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:50.685406 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q65s9_ba1b3401-8894-4b43-8ab1-58fe4d6ee017/kube-rbac-proxy/0.log" Jun 11 23:14:50.920749 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:50.920671 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nmkb6_149874b1-5504-4544-a52f-c69bf66ee3b3/dns-node-resolver/0.log" Jun 11 23:14:51.550748 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:51.550711 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p2zlk_2c595b8d-0a5e-49c3-bf2f-9fdfba4e2751/node-ca/0.log" Jun 11 23:14:52.645099 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:52.645070 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-bfm8m_9994305c-64de-4b66-a50b-041b96c1d7a8/discovery/0.log" Jun 11 23:14:52.679213 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:52.679187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c495b869c-6khwv_4226b38a-6127-4aa7-8dc8-4122d4f2b076/kube-auth-proxy/0.log" Jun 11 23:14:52.868320 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:52.868297 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-j7567_b28e4b34-231c-4707-9c2f-1ae8cff76bfe/payload-pre-processing/0.log" Jun 11 23:14:52.921792 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:52.921700 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-x78k4_eae48b67-0f65-4019-85bc-b2e4f0b80dd1/payload-processing/0.log" Jun 11 23:14:53.680384 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:53.680343 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fcl8g_4643df29-4bfb-4cf1-ba2b-226430b8f802/serve-healthcheck-canary/0.log" Jun 11 23:14:54.340914 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:54.340891 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gnbfj_2f5fee67-6a24-476f-a02c-16d9082640ff/kube-rbac-proxy/0.log" Jun 11 23:14:54.367950 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:54.367917 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gnbfj_2f5fee67-6a24-476f-a02c-16d9082640ff/exporter/0.log" Jun 11 23:14:54.397457 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:54.397421 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gnbfj_2f5fee67-6a24-476f-a02c-16d9082640ff/extractor/0.log" Jun 11 23:14:55.388078 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:55.388054 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b88td/perf-node-gather-daemonset-j248b" Jun 11 23:14:56.871449 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:56.871420 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-784f85db5-xpwpm_9d75a68a-0e53-4313-b1b2-a615a90457bb/nginx/0.log" Jun 11 23:14:56.910128 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:56.910102 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-zfcqp_6e29b810-7f3c-4bb7-a99a-8d4b4e6eb098/manager/0.log" Jun 11 23:14:56.950401 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:56.950380 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-w7hzg_aa2f432d-c676-4ef2-8053-770bebc15650/manager/0.log" Jun 11 23:14:56.976012 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:56.975990 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-59dcb9b5b7-cxb2j_d64aabc1-d855-4f8f-9824-dd5987e55812/maas-api/0.log" Jun 11 23:14:57.014589 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:57.014562 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85b78b4694-x727z_97cab4ae-d4e1-4021-85a5-3953e2c5ab35/manager/0.log" Jun 11 23:14:57.036117 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:57.036096 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-lxw9r_4f36dab2-ccb3-4541-99c2-f997adcf7536/server/0.log" Jun 11 23:14:57.083018 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:57.082996 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-mxmkn_0a853ddc-b4a3-4cdf-a408-bb138034e80d/manager/0.log" Jun 11 23:14:57.113153 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:57.113124 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f864d649-2hgvp_96b109cf-c44b-4a83-9130-22a6cc268ff5/manager/0.log" Jun 11 23:14:57.134102 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:57.134038 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f864d649-2hgvp_96b109cf-c44b-4a83-9130-22a6cc268ff5/copy-manifests/0.log" Jun 11 23:14:57.296424 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:57.296401 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mg8pr_d2babdef-8877-472a-b01c-a8449aaac9e0/postgres/0.log" Jun 11 23:14:59.182343 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:14:59.182311 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-h4bsv_4f3a2c3e-4e22-4fff-88c1-1b3513d37ece/openshift-lws-operator/0.log" Jun 11 23:15:00.154895 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.154797 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29686995-tzdkr"] Jun 11 23:15:00.158393 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.158375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" Jun 11 23:15:00.174774 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.174728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k87rw\" (UniqueName: \"kubernetes.io/projected/4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60-kube-api-access-k87rw\") pod \"maas-api-key-cleanup-29686995-tzdkr\" (UID: \"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60\") " pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" Jun 11 23:15:00.175656 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.175636 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29686995-tzdkr"] Jun 11 23:15:00.276042 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.276009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k87rw\" (UniqueName: \"kubernetes.io/projected/4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60-kube-api-access-k87rw\") pod \"maas-api-key-cleanup-29686995-tzdkr\" (UID: \"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60\") " pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" Jun 11 23:15:00.294501 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.294480 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k87rw\" (UniqueName: \"kubernetes.io/projected/4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60-kube-api-access-k87rw\") pod \"maas-api-key-cleanup-29686995-tzdkr\" (UID: \"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60\") " pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" Jun 11 23:15:00.473784 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.473754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" Jun 11 23:15:00.609266 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.609237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29686995-tzdkr"] Jun 11 23:15:00.611002 ip-10-0-138-236 kubenswrapper[2562]: W0611 23:15:00.610976 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbdae45_fd8c_4abf_bdf7_0c57c04e5f60.slice/crio-9e5516a7e7c209d958a5311bd074ddd5a90307bf827d70333c018ee2642d98f8 WatchSource:0}: Error finding container 9e5516a7e7c209d958a5311bd074ddd5a90307bf827d70333c018ee2642d98f8: Status 404 returned error can't find the container with id 9e5516a7e7c209d958a5311bd074ddd5a90307bf827d70333c018ee2642d98f8 Jun 11 23:15:00.612884 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:00.612866 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 23:15:01.436306 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:01.436263 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" event={"ID":"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60","Type":"ContainerStarted","Data":"9e5516a7e7c209d958a5311bd074ddd5a90307bf827d70333c018ee2642d98f8"} Jun 11 23:15:02.958159 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:02.958116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-bfm8m_9994305c-64de-4b66-a50b-041b96c1d7a8/discovery/0.log" Jun 11 23:15:02.967435 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:02.967404 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c495b869c-6khwv_4226b38a-6127-4aa7-8dc8-4122d4f2b076/kube-auth-proxy/0.log" Jun 11 23:15:03.015861 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:03.015750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-j7567_b28e4b34-231c-4707-9c2f-1ae8cff76bfe/payload-pre-processing/0.log" Jun 11 23:15:03.030402 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:03.030376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-x78k4_eae48b67-0f65-4019-85bc-b2e4f0b80dd1/payload-processing/0.log" Jun 11 23:15:03.448485 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:03.448449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" event={"ID":"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60","Type":"ContainerStarted","Data":"c60d4bfde03c5a2dd4c1c761bf27d428ad956ace358290c4b73ba1a3bf33c5d1"} Jun 11 23:15:03.466971 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:03.466923 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" podStartSLOduration=1.328434347 podStartE2EDuration="3.466907986s" totalImagesPullingTime="2.138473639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 23:15:03.464740184 +0000 UTC m=+1355.409989078" watchObservedRunningTime="2026-06-11 23:15:03.466907986 +0000 UTC m=+1355.412156876" Jun 11 23:15:04.201329 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:04.201300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-bfm8m_9994305c-64de-4b66-a50b-041b96c1d7a8/discovery/0.log" Jun 11 23:15:04.216214 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:04.216181 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c495b869c-6khwv_4226b38a-6127-4aa7-8dc8-4122d4f2b076/kube-auth-proxy/0.log" Jun 11 23:15:04.269695 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:04.269669 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-j7567_b28e4b34-231c-4707-9c2f-1ae8cff76bfe/payload-pre-processing/0.log" Jun 11 23:15:04.289780 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:04.289751 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-x78k4_eae48b67-0f65-4019-85bc-b2e4f0b80dd1/payload-processing/0.log" Jun 11 23:15:04.455275 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:04.455183 2562 generic.go:356] "Generic (PLEG): container finished" podID="4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60" containerID="c60d4bfde03c5a2dd4c1c761bf27d428ad956ace358290c4b73ba1a3bf33c5d1" exitCode=0 Jun 11 23:15:04.455444 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:04.455271 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" event={"ID":"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60","Type":"ContainerDied","Data":"c60d4bfde03c5a2dd4c1c761bf27d428ad956ace358290c4b73ba1a3bf33c5d1"} Jun 11 23:15:05.613752 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:05.613723 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686995-tzdkr" Jun 11 23:15:05.724750 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:05.724659 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k87rw\" (UniqueName: \"kubernetes.io/projected/4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60-kube-api-access-k87rw\") pod \"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60\" (UID: \"4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60\") " Jun 11 23:15:05.726659 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:05.726632 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60-kube-api-access-k87rw" (OuterVolumeSpecName: "kube-api-access-k87rw") pod "4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60" (UID: "4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60"). InnerVolumeSpecName "kube-api-access-k87rw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 23:15:05.825253 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:05.825223 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k87rw\" (UniqueName: \"kubernetes.io/projected/4bbdae45-fd8c-4abf-bdf7-0c57c04e5f60-kube-api-access-k87rw\") on node \"ip-10-0-138-236.ec2.internal\" DevicePath \"\"" Jun 11 23:15:06.064041 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:06.063963 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4qvkk_2804aac5-78a3-4aa8-a1e3-6dcc0b17d95e/kube-multus/0.log" Jun 11 23:15:06.115313 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:06.115285 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-bfm8m_9994305c-64de-4b66-a50b-041b96c1d7a8/discovery/0.log" Jun 11 23:15:06.125923 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:06.125900 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c495b869c-6khwv_4226b38a-6127-4aa7-8dc8-4122d4f2b076/kube-auth-proxy/0.log" Jun 11 23:15:06.166895 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:06.166864 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-j7567_b28e4b34-231c-4707-9c2f-1ae8cff76bfe/payload-pre-processing/0.log" Jun 11 23:15:06.181025 ip-10-0-138-236 kubenswrapper[2562]: I0611 23:15:06.180999 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-x78k4_eae48b67-0f65-4019-85bc-b2e4f0b80dd1/payload-processing/0.log"