Jun 09 05:28:46.178696 ip-10-0-135-121 systemd[1]: Starting Kubernetes Kubelet... Jun 09 05:28:46.628433 ip-10-0-135-121 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 05:28:46.628433 ip-10-0-135-121 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 05:28:46.628433 ip-10-0-135-121 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 05:28:46.628433 ip-10-0-135-121 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 05:28:46.628433 ip-10-0-135-121 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 05:28:46.629606 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.629473 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 05:28:46.633504 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633488 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 05:28:46.633504 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633502 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633507 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633510 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633514 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633517 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633520 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633528 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633531 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633534 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633537 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633540 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633542 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633545 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633547 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633550 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633553 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633555 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633558 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633561 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 05:28:46.633567 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633564 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633567 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633569 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633572 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633575 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633578 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633580 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633583 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633586 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633588 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633591 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633593 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633597 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633599 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633602 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633604 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633607 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633609 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633612 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633617 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 05:28:46.634032 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633620 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633622 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633625 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633627 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633630 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633632 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633635 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633638 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633640 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633643 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633646 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633648 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633651 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633654 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633657 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633661 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633665 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633668 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633671 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633674 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 05:28:46.634553 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633676 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633679 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633681 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633683 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633687 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633691 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633693 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633696 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633698 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633701 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633703 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633705 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633708 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633710 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633714 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633717 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633719 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633722 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633724 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633726 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 05:28:46.635054 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633729 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633731 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633734 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633736 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633739 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633741 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633745 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633748 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633750 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633753 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633757 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633760 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633763 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633766 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633768 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633771 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633774 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633776 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.633779 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 05:28:46.635544 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634474 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634481 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634484 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634487 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634489 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634492 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634502 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634505 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634508 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634510 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634513 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634516 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634525 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634528 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634531 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634533 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634536 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634539 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634541 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634545 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 05:28:46.636013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634548 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634551 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634554 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634556 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634560 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634562 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634565 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634568 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634571 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634573 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634576 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634578 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634581 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634583 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634586 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634589 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634591 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634594 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634596 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634600 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 05:28:46.636529 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634602 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634605 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634607 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634610 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634613 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634616 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634618 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634621 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634623 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634626 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634628 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634632 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634634 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634637 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634639 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634642 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634645 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634647 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634650 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634652 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 05:28:46.637020 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634655 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634657 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634660 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634664 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634668 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634671 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634673 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634676 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634678 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634681 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634684 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634688 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634693 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634696 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634699 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634702 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634705 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634709 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634712 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 05:28:46.637531 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634715 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634717 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634722 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634726 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634730 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634733 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634736 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634738 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634741 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634744 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634747 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634749 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634752 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634755 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634757 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634760 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634762 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634765 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634767 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 05:28:46.638023 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.634770 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635113 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635118 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635121 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635123 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635126 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635129 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635132 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635135 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635138 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635141 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635144 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635147 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635149 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635152 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635154 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635157 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635160 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635163 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635165 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 05:28:46.638516 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635168 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635170 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635172 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635175 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635178 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635180 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635183 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635186 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635188 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635191 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635193 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635195 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635198 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635200 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635203 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635205 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635208 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635211 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635213 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635218 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 05:28:46.639013 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635221 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635223 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635226 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635230 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635233 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635236 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635239 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635242 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635245 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635247 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635250 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635253 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635255 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635257 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635260 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635263 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635266 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635269 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635271 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 05:28:46.639557 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635273 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635276 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635279 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635281 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635284 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635287 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635290 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635292 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635294 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635297 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635300 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635303 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635305 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635308 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635311 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635314 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635316 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635319 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635321 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635324 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 05:28:46.640022 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635328 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635331 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635333 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635336 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635339 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635341 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635344 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635346 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635349 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635351 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635354 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635356 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635359 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635361 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635364 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635366 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635370 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635374 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635377 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635380 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 05:28:46.640563 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.635397 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635466 2567 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635476 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635483 2567 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635487 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635491 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635495 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635499 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635503 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635506 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635509 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635513 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635516 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635519 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635522 2567 flags.go:64] FLAG: --cgroup-root="" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635525 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635528 2567 flags.go:64] FLAG: --client-ca-file="" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635530 2567 flags.go:64] FLAG: --cloud-provider="external" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635533 2567 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635536 2567 flags.go:64] FLAG: --cluster-domain="" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635539 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635542 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635545 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635549 2567 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 05:28:46.641060 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635552 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635556 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635559 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635562 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635565 2567 flags.go:64] FLAG: --contention-profiling="false" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635569 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635571 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635575 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635577 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635581 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635584 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635587 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635590 2567 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635593 2567 flags.go:64] FLAG: --enable-server="true" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635596 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635601 2567 flags.go:64] FLAG: --event-burst="100" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635604 2567 flags.go:64] FLAG: --event-qps="50" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635607 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635610 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635612 2567 flags.go:64] FLAG: --eviction-hard="" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635616 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635619 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635622 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635625 2567 flags.go:64] FLAG: --eviction-soft="" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635627 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 05:28:46.641657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635630 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635633 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635636 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635639 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635642 2567 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635644 2567 flags.go:64] FLAG: --feature-gates="" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635648 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635651 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635654 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635657 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635660 2567 flags.go:64] FLAG: --healthz-port="10248" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635663 2567 flags.go:64] FLAG: --help="false" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635666 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635669 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635672 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635674 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635681 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635685 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635687 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635690 2567 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635693 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635697 2567 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635699 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635703 2567 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 05:28:46.642263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635706 2567 flags.go:64] FLAG: --kube-reserved="" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635709 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635712 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635715 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635717 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635720 2567 flags.go:64] FLAG: --lock-file="" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635723 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635725 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635728 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635734 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635736 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635739 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635742 2567 flags.go:64] FLAG: --logging-format="text" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635745 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635748 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635751 2567 flags.go:64] FLAG: --manifest-url="" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635754 2567 flags.go:64] FLAG: --manifest-url-header="" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635758 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635761 2567 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635764 2567 flags.go:64] FLAG: --max-pods="110" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635769 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635772 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635775 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635778 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635782 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 05:28:46.643004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635785 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635788 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635794 2567 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635797 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635800 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635803 2567 flags.go:64] FLAG: --pod-cidr="" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635806 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635812 2567 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635815 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635818 2567 flags.go:64] FLAG: --pods-per-core="0" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635821 2567 flags.go:64] FLAG: --port="10250" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635824 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635827 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe47fa22ed860376" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635830 2567 flags.go:64] FLAG: --qos-reserved="" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635833 2567 flags.go:64] FLAG: --read-only-port="10255" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635836 2567 flags.go:64] FLAG: --register-node="true" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635838 2567 flags.go:64] FLAG: --register-with-taints="" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635842 2567 flags.go:64] FLAG: --registry-burst="10" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635845 2567 flags.go:64] FLAG: --registry-qps="5" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635847 2567 flags.go:64] FLAG: --reserved-cpus="" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635850 2567 flags.go:64] FLAG: --reserved-memory="" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635853 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635856 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635859 2567 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635862 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 05:28:46.643959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635865 2567 flags.go:64] FLAG: --runonce="false" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635867 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635872 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635875 2567 flags.go:64] FLAG: --seccomp-default="false" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635878 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635881 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635884 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635889 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635892 2567 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635895 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635898 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635900 2567 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635903 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635906 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635909 2567 flags.go:64] FLAG: --system-cgroups="" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635912 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635918 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635921 2567 flags.go:64] FLAG: --tls-cert-file="" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635923 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635926 2567 flags.go:64] FLAG: --tls-min-version="" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635929 2567 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635932 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635935 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635938 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635941 2567 flags.go:64] FLAG: --v="2" Jun 09 05:28:46.644604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635945 2567 flags.go:64] FLAG: --version="false" Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635948 2567 flags.go:64] FLAG: --vmodule="" Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635952 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.635955 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636066 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636070 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636073 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636076 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636079 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636085 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636088 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636091 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636093 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636096 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636100 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636102 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636105 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636107 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636110 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636113 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 05:28:46.645229 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636115 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636119 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636123 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636126 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636129 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636132 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636135 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636138 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636141 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636143 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636146 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636149 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636151 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636154 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636156 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636159 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636161 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636164 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636167 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636171 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 05:28:46.645743 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636174 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636179 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636182 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636185 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636187 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636190 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636194 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636197 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636200 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636204 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636207 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636210 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636213 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636215 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636218 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636220 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636224 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636227 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636230 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 05:28:46.646239 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636232 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636235 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636237 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636240 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636242 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636245 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636247 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636249 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636252 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636255 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636257 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636260 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636262 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636264 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636268 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636271 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636273 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636276 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636278 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 05:28:46.646817 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636282 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636284 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636287 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636289 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636292 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636295 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636297 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636300 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636302 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636305 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636307 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636647 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636651 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636654 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636656 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636659 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636662 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636664 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636667 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636669 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 05:28:46.647295 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636672 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636674 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636677 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636679 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.636682 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.636687 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.644570 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.644586 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.645493 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.645505 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645565 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645569 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645572 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645575 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 05:28:46.647797 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645578 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645582 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645586 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645589 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645593 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645596 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645599 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645602 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645605 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645608 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645611 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645614 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645616 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645619 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645622 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645625 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645628 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645631 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645634 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645636 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 05:28:46.648176 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645639 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645642 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645644 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645647 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645650 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645654 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645657 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645659 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645663 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645667 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645670 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645673 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645676 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645679 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645682 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645685 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645687 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645690 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645693 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 05:28:46.648664 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645695 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645698 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645700 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645703 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645706 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645708 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645711 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645714 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645716 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645719 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645721 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645724 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645726 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645729 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645731 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645734 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645736 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645739 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645742 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 05:28:46.649122 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645745 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645748 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645750 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645753 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645756 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645759 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645761 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645764 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645767 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645769 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645772 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645775 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645777 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645780 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645782 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645785 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645787 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645791 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645793 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645796 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 05:28:46.649679 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645798 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645801 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645803 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645806 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645809 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645811 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645814 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645817 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645819 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645822 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645826 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645831 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645834 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645837 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645840 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645842 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 05:28:46.650164 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645845 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.645850 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645959 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645965 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645968 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645971 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645974 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645977 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645980 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645983 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645985 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645988 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645990 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645993 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645996 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 05:28:46.650592 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.645998 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646001 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646003 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646006 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646009 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646011 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646014 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646016 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646019 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646021 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646024 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646027 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646029 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646032 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646036 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646040 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646043 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646046 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 05:28:46.651014 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646049 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646052 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646055 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646058 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646061 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646063 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646066 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646068 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646071 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646074 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646076 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646079 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646082 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646084 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646087 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646091 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646094 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646097 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646100 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646103 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 05:28:46.651523 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646106 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646109 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646111 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646114 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646117 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646120 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646123 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646125 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646128 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646130 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646133 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646136 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646138 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646141 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646143 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646146 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646148 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646151 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646153 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646156 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 05:28:46.651993 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646158 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646161 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646163 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646166 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646168 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646171 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646173 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646177 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646179 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646181 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646184 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646186 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646189 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646191 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646194 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646196 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646199 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646202 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646204 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646207 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 05:28:46.652488 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646209 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646212 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646214 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646217 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646219 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646222 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646224 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:46.646227 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.646232 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.646332 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.648749 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.650370 2567 server.go:1019] "Starting client certificate rotation" Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.650484 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 05:28:46.652973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.650527 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 05:28:46.654600 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.654586 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 05:28:46.655740 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.655722 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 05:28:46.670607 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.670584 2567 log.go:25] "Validated CRI v1 runtime API" Jun 09 05:28:46.673206 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.673191 2567 log.go:25] "Validated CRI v1 image API" Jun 09 05:28:46.674069 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.674053 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 05:28:46.679709 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.679691 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 05:28:46.680829 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.680811 2567 fs.go:135] Filesystem UUIDs: map[4ab39297-ce67-457e-a59d-3b1a7343d1e9:/dev/nvme0n1p3 4db6e38c-bfe6-426c-86bd-3aeba78849c5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 09 05:28:46.680881 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.680838 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 05:28:46.686610 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.686467 2567 manager.go:216] Machine: {Timestamp:2026-06-09 05:28:46.685039149 +0000 UTC m=+0.379382138 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100106 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ebac2dd617f9f841a455ab8de4e36 SystemUUID:ec2ebac2-dd61-7f9f-841a-455ab8de4e36 BootID:e0d8ca97-f036-44b6-adbb-218b7d45a0df Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:ef:08:51:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:ef:08:51:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:37:43:f5:93:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 05:28:46.686610 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.686606 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 05:28:46.686727 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.686680 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 05:28:46.689219 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.689198 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 05:28:46.689869 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.689229 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-121.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 05:28:46.690562 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.690543 2567 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 05:28:46.690637 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.690590 2567 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 05:28:46.690674 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.690616 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 05:28:46.690880 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.690700 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 05:28:46.691899 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.691887 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 05:28:46.696029 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696017 2567 state_mem.go:36] "Initialized new in-memory state store" Jun 09 05:28:46.696178 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696169 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 05:28:46.696208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696200 2567 kubelet.go:486] "Attempting to sync node with API server" Jun 09 05:28:46.696235 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696210 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 05:28:46.696235 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696224 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 05:28:46.696235 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696234 2567 kubelet.go:398] "Adding apiserver pod source" Jun 09 05:28:46.696325 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696242 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 05:28:46.696665 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696652 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 05:28:46.696703 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.696672 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 05:28:46.699097 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.699080 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 05:28:46.700765 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.700750 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 05:28:46.700841 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.700788 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 05:28:46.702124 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702099 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 05:28:46.702124 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702118 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702128 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702136 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702147 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702156 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702166 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702175 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702185 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702193 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702205 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 05:28:46.702257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.702213 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 05:28:46.703229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.703218 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 05:28:46.703278 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.703231 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 05:28:46.706369 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.706353 2567 server.go:1268] "Started kubelet" Jun 09 05:28:46.706552 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.706503 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 05:28:46.706603 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.706572 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 05:28:46.707101 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.707078 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 05:28:46.707208 ip-10-0-135-121 systemd[1]: Started Kubernetes Kubelet. Jun 09 05:28:46.707720 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.707621 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-121.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 05:28:46.707720 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.707672 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 05:28:46.707977 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.707839 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-121.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 05:28:46.708667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.708651 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 05:28:46.709113 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.709101 2567 server.go:310] "Adding debug handlers to kubelet server" Jun 09 05:28:46.715649 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.714498 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-121.ec2.internal.18b75289295db6eb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-121.ec2.internal,UID:ip-10-0-135-121.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-121.ec2.internal,},FirstTimestamp:2026-06-09 05:28:46.706325227 +0000 UTC m=+0.400668198,LastTimestamp:2026-06-09 05:28:46.706325227 +0000 UTC m=+0.400668198,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-121.ec2.internal,}" Jun 09 05:28:46.715747 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.715718 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 05:28:46.717424 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.717408 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 05:28:46.717520 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.717408 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 05:28:46.717634 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.717618 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:46.718163 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718134 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 05:28:46.718163 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718151 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 05:28:46.718303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718230 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 05:28:46.718303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718281 2567 reconstruct.go:97] "Volume reconstruction finished" Jun 09 05:28:46.718303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718291 2567 reconciler.go:29] "Reconciler: start to sync state" Jun 09 05:28:46.718437 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718417 2567 factory.go:55] Registering systemd factory Jun 09 05:28:46.718437 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718437 2567 factory.go:223] Registration of the systemd container factory successfully Jun 09 05:28:46.718693 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718680 2567 factory.go:153] Registering CRI-O factory Jun 09 05:28:46.718734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718696 2567 factory.go:223] Registration of the crio container factory successfully Jun 09 05:28:46.718763 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718748 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 05:28:46.718793 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718777 2567 factory.go:103] Registering Raw factory Jun 09 05:28:46.718793 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.718792 2567 manager.go:1196] Started watching for new ooms in manager Jun 09 05:28:46.719426 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.719412 2567 manager.go:349] Starting recovery of all containers Jun 09 05:28:46.722408 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.722368 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fdbcr" Jun 09 05:28:46.728645 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.728615 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 09 05:28:46.728746 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.728602 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-121.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 09 05:28:46.729441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.729420 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fdbcr" Jun 09 05:28:46.731215 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.731198 2567 manager.go:354] Recovery completed Jun 09 05:28:46.735501 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.735400 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.737740 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.737725 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.737805 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.737752 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.737805 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.737761 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.738255 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.738243 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 05:28:46.738255 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.738254 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 05:28:46.738330 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.738268 2567 state_mem.go:36] "Initialized new in-memory state store" Jun 09 05:28:46.738330 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.738300 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.739013 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.738999 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.739083 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.739022 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.739083 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.739032 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.740063 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.740052 2567 policy_none.go:49] "None policy: Start" Jun 09 05:28:46.740110 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.740067 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 05:28:46.740110 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.740076 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 05:28:46.742250 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.742240 2567 policy_none.go:47] "Start" Jun 09 05:28:46.778025 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778007 2567 manager.go:337] "Starting Device Plugin manager" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.778043 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778057 2567 server.go:87] "Starting device plugin registration server" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778501 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778512 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778601 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778677 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.778909 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.779140 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.779193 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.791470 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.792666 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.792711 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.792736 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.792776 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 05:28:46.802799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.795412 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 05:28:46.879441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.879368 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.881326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.881311 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.881440 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.881348 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.881440 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.881369 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.881440 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.881425 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.890464 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.890445 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.890513 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.890469 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-121.ec2.internal\": node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:46.893428 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.893412 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal"] Jun 09 05:28:46.893485 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.893474 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.894420 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.894405 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.894510 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.894434 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.894510 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.894450 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.897036 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897022 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.897198 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897184 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.897245 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897216 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.897750 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897735 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.897816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897752 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.897816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897758 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.897816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897768 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.897816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897770 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.897816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.897782 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.900134 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.900119 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.900201 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.900158 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 05:28:46.900831 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.900815 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientMemory" Jun 09 05:28:46.900908 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.900845 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 05:28:46.900908 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.900859 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeHasSufficientPID" Jun 09 05:28:46.916442 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.916419 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:46.919587 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.919572 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fdafb58a07221d574c211454336b738-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal\" (UID: \"8fdafb58a07221d574c211454336b738\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.919650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.919600 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fdafb58a07221d574c211454336b738-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal\" (UID: \"8fdafb58a07221d574c211454336b738\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.919650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:46.919628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7bbdc86751d54a40c4e45022ba51be3b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-121.ec2.internal\" (UID: \"7bbdc86751d54a40c4e45022ba51be3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.924984 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.924969 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-121.ec2.internal\" not found" node="ip-10-0-135-121.ec2.internal" Jun 09 05:28:46.929706 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:46.929689 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-121.ec2.internal\" not found" node="ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.016874 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.016831 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.020112 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.020095 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fdafb58a07221d574c211454336b738-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal\" (UID: \"8fdafb58a07221d574c211454336b738\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.020183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.020120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fdafb58a07221d574c211454336b738-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal\" (UID: \"8fdafb58a07221d574c211454336b738\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.020183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.020137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7bbdc86751d54a40c4e45022ba51be3b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-121.ec2.internal\" (UID: \"7bbdc86751d54a40c4e45022ba51be3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.020183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.020175 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7bbdc86751d54a40c4e45022ba51be3b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-121.ec2.internal\" (UID: \"7bbdc86751d54a40c4e45022ba51be3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.020280 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.020179 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fdafb58a07221d574c211454336b738-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal\" (UID: \"8fdafb58a07221d574c211454336b738\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.020280 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.020202 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fdafb58a07221d574c211454336b738-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal\" (UID: \"8fdafb58a07221d574c211454336b738\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.117301 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.117271 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.217939 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.217864 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.231269 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.231242 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.236840 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.236824 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.318703 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.318662 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.419116 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.419091 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.519576 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.519501 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.620056 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.620025 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.649643 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.649608 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 05:28:47.650166 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.649814 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 05:28:47.720095 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.720068 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.733277 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.733232 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 05:23:46 +0000 UTC" deadline="2028-01-20 22:55:53.79681289 +0000 UTC" Jun 09 05:28:47.733277 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.733273 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14177h27m6.063543327s" Jun 09 05:28:47.763008 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:47.762981 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fdafb58a07221d574c211454336b738.slice/crio-80663bce866215614dcfd100f60dd763e2e1c239d107972dea3c092bd9685aab WatchSource:0}: Error finding container 80663bce866215614dcfd100f60dd763e2e1c239d107972dea3c092bd9685aab: Status 404 returned error can't find the container with id 80663bce866215614dcfd100f60dd763e2e1c239d107972dea3c092bd9685aab Jun 09 05:28:47.763448 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:47.763432 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbdc86751d54a40c4e45022ba51be3b.slice/crio-5ec0e588dc589dcfe1c28bb48e38c746dfbd32f423dda83bf252a556b4c56c78 WatchSource:0}: Error finding container 5ec0e588dc589dcfe1c28bb48e38c746dfbd32f423dda83bf252a556b4c56c78: Status 404 returned error can't find the container with id 5ec0e588dc589dcfe1c28bb48e38c746dfbd32f423dda83bf252a556b4c56c78 Jun 09 05:28:47.770559 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.770540 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 05:28:47.795599 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.795551 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" event={"ID":"7bbdc86751d54a40c4e45022ba51be3b","Type":"ContainerStarted","Data":"5ec0e588dc589dcfe1c28bb48e38c746dfbd32f423dda83bf252a556b4c56c78"} Jun 09 05:28:47.796554 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.796533 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" event={"ID":"8fdafb58a07221d574c211454336b738","Type":"ContainerStarted","Data":"80663bce866215614dcfd100f60dd763e2e1c239d107972dea3c092bd9685aab"} Jun 09 05:28:47.811952 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.811932 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 05:28:47.821086 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:47.821064 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-121.ec2.internal\" not found" Jun 09 05:28:47.893759 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.893730 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 05:28:47.917702 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.917682 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.923978 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.923958 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 05:28:47.924844 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.924830 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 05:28:47.925887 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.925874 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" Jun 09 05:28:47.935983 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:47.935967 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 05:28:48.697497 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.697464 2567 apiserver.go:52] "Watching apiserver" Jun 09 05:28:48.705452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.705426 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 05:28:48.705905 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.705875 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr","openshift-cluster-node-tuning-operator/tuned-4mhlm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal","openshift-multus/multus-jlzjg","openshift-multus/network-metrics-daemon-wfj72","openshift-network-diagnostics/network-check-target-6kwjn","kube-system/konnectivity-agent-4j88l","openshift-dns/node-resolver-l6qxp","openshift-image-registry/node-ca-vqxk4","openshift-multus/multus-additional-cni-plugins-mcm2j","openshift-network-operator/iptables-alerter-zcpl5","openshift-ovn-kubernetes/ovnkube-node-jspp7","kube-system/global-pull-secret-syncer-28hwr","kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal"] Jun 09 05:28:48.708798 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.708772 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:48.708905 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.708864 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:28:48.711012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.710823 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.713023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.712999 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.713128 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.713110 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vn79s\"" Jun 09 05:28:48.714055 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.713835 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.714626 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.714577 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.715125 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.715106 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:48.715195 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.715170 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:28:48.715257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.715239 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.715484 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.715466 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 05:28:48.715558 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.715484 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c85v9\"" Jun 09 05:28:48.715558 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.715502 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.717565 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.717302 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.717939 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.717894 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 05:28:48.720008 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.719977 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 05:28:48.720094 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.720036 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 05:28:48.720156 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.720108 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.722849 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.722243 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.725153 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.724692 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.727076 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727057 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.727218 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727198 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-daemon-config\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727288 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727233 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-run\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727288 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727259 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3d44367-adf6-4045-b571-8936e08e952b-konnectivity-ca\") pod \"konnectivity-agent-4j88l\" (UID: \"f3d44367-adf6-4045-b571-8936e08e952b\") " pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.727288 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727283 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/601e07b5-4e8e-4935-8288-0be80cac7a3b-cni-binary-copy\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727472 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727306 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-k8s-cni-cncf-io\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727472 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-systemd\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727472 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-lib-modules\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727472 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727422 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-var-lib-kubelet\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727472 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68552\" (UniqueName: \"kubernetes.io/projected/ba81a4c9-7047-466e-b774-30b32290ad01-kube-api-access-68552\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727472 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3d44367-adf6-4045-b571-8936e08e952b-agent-certs\") pod \"konnectivity-agent-4j88l\" (UID: \"f3d44367-adf6-4045-b571-8936e08e952b\") " pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-system-cni-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-netns\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysctl-conf\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727583 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba81a4c9-7047-466e-b774-30b32290ad01-etc-tuned\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727606 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-socket-dir-parent\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-cni-bin\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727652 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-hostroot\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727678 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-modprobe-d\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727703 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npfj4\" (UniqueName: \"kubernetes.io/projected/1f00d08c-a113-479f-a8d3-b8ef63144cc0-kube-api-access-npfj4\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727726 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-kubelet\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.727757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727751 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-multus-certs\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727774 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-registration-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727797 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-sys-fs\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727824 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvzn5\" (UniqueName: \"kubernetes.io/projected/601e07b5-4e8e-4935-8288-0be80cac7a3b-kube-api-access-bvzn5\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727849 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-sys\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba81a4c9-7047-466e-b774-30b32290ad01-tmp\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727892 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-cnibin\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-os-release\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727936 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-conf-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727959 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-etc-kubernetes\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.727986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z7xh\" (UniqueName: \"kubernetes.io/projected/879b1a42-fd63-4e8a-be9b-7886e17a8408-kube-api-access-4z7xh\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728008 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-kubernetes\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728030 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-host\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728051 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-socket-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728074 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-device-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728096 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728120 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysconfig\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.728309 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728146 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysctl-d\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.729238 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728169 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-etc-selinux\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.729238 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728193 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:48.729238 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-cni-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.729238 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.728239 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-cni-multus\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.729553 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.729535 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.732155 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.732063 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.734575 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.734557 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.734662 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.734621 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:28:48.737005 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.736985 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6d9n7\"" Jun 09 05:28:48.737358 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.737340 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 05:28:48.737758 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.737737 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.738059 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.738028 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.738255 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.738240 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wznqz\"" Jun 09 05:28:48.738447 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.738431 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 05:28:48.738686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.738671 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9gbrz\"" Jun 09 05:28:48.738943 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.738922 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.740653 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.740638 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.741658 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.741637 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 05:28:48.744199 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.744178 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.744441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.744309 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 05:28:48.745142 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.745013 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.745142 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.745076 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 05:28:48.745142 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.745095 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.745595 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.745452 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 05:28:48.745595 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.745488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dbp9f\"" Jun 09 05:28:48.745595 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.745497 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.746171 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.746037 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 05:28:48.746171 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.746057 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 05:28:48.746171 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.746064 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-prrtd\"" Jun 09 05:28:48.746363 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.746176 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 05:28:48.746625 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.746606 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 05:28:48.746895 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.746875 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d6nrj\"" Jun 09 05:28:48.747105 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.747086 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 05:28:48.747166 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.747092 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sdwt8\"" Jun 09 05:28:48.747368 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.747351 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 05:28:48.754310 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.754288 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 05:28:48.779319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.779279 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgqsj" Jun 09 05:28:48.788450 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.788426 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgqsj" Jun 09 05:28:48.819131 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.819097 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 05:28:48.828763 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828737 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-sys\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.828894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828769 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-os-release\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.828894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828786 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-conf-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.828894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828811 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-etc-kubernetes\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.828894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828840 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovn-node-metrics-cert\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.828894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828870 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsrjd\" (UniqueName: \"kubernetes.io/projected/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-kube-api-access-dsrjd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.828894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828892 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z7xh\" (UniqueName: \"kubernetes.io/projected/879b1a42-fd63-4e8a-be9b-7886e17a8408-kube-api-access-4z7xh\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828907 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-host\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-device-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828951 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828970 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-systemd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.828990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysconfig\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829003 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysctl-d\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-etc-selinux\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829038 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-cni-multus\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829080 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-run\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829103 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3d44367-adf6-4045-b571-8936e08e952b-konnectivity-ca\") pod \"konnectivity-agent-4j88l\" (UID: \"f3d44367-adf6-4045-b571-8936e08e952b\") " pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829126 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/601e07b5-4e8e-4935-8288-0be80cac7a3b-cni-binary-copy\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829149 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqlbt\" (UniqueName: \"kubernetes.io/projected/1560f09c-5460-43eb-b87e-a22e744e1cd4-kube-api-access-gqlbt\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.829192 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829176 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-systemd\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68552\" (UniqueName: \"kubernetes.io/projected/ba81a4c9-7047-466e-b774-30b32290ad01-kube-api-access-68552\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-system-cni-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829247 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-netns\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5af69f3-c1ad-4531-b392-4313706bcb16-serviceca\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829291 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkdc5\" (UniqueName: \"kubernetes.io/projected/1b6e3541-7a83-41ea-bdea-f8e2f4787005-kube-api-access-vkdc5\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829312 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-etc-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829333 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-ovn\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829352 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysconfig\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829473 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-systemd\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829519 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysctl-conf\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829723 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-system-cni-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829736 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-conf-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829771 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-netns\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829783 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-etc-kubernetes\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829838 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-device-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829862 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-cni-multus\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.829906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829862 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3d44367-adf6-4045-b571-8936e08e952b-konnectivity-ca\") pod \"konnectivity-agent-4j88l\" (UID: \"f3d44367-adf6-4045-b571-8936e08e952b\") " pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829912 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-host\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829949 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-run\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829364 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysctl-conf\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829976 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/601e07b5-4e8e-4935-8288-0be80cac7a3b-cni-binary-copy\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.829996 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-socket-dir-parent\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830021 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-sys\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-cni-bin\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830045 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-sysctl-d\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830063 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-cni-bin\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830074 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-os-release\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-hostroot\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830108 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5af69f3-c1ad-4531-b392-4313706bcb16-host\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830070 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-hostroot\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1560f09c-5460-43eb-b87e-a22e744e1cd4-iptables-alerter-script\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830052 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-etc-selinux\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-node-log\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830173 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-socket-dir-parent\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.830594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830178 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovnkube-script-lib\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830226 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-modprobe-d\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830255 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-registration-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvzn5\" (UniqueName: \"kubernetes.io/projected/601e07b5-4e8e-4935-8288-0be80cac7a3b-kube-api-access-bvzn5\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830333 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1560f09c-5460-43eb-b87e-a22e744e1cd4-host-slash\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830365 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-modprobe-d\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830396 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830423 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba81a4c9-7047-466e-b774-30b32290ad01-tmp\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-cnibin\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830503 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/345979f3-c948-4fc6-87e0-b7e7ac3242e5-dbus\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-kubernetes\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-socket-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-systemd-units\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.831490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830629 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-cnibin\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-env-overrides\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830683 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-registration-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830714 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-etc-kubernetes\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830727 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830784 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-cni-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830806 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-daemon-config\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830807 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-socket-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830832 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-cnibin\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.830954 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-cni-dir\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.831050 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831081 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-cni-netd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831048 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.831145 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:49.331117839 +0000 UTC m=+3.025460811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831247 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/601e07b5-4e8e-4935-8288-0be80cac7a3b-multus-daemon-config\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831289 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-k8s-cni-cncf-io\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831316 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1b6e3541-7a83-41ea-bdea-f8e2f4787005-hosts-file\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.832311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831363 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-k8s-cni-cncf-io\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831402 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-os-release\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831438 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-lib-modules\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-var-lib-kubelet\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831549 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-var-lib-kubelet\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831580 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba81a4c9-7047-466e-b774-30b32290ad01-lib-modules\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831616 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3d44367-adf6-4045-b571-8936e08e952b-agent-certs\") pod \"konnectivity-agent-4j88l\" (UID: \"f3d44367-adf6-4045-b571-8936e08e952b\") " pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831661 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5l25\" (UniqueName: \"kubernetes.io/projected/a5af69f3-c1ad-4531-b392-4313706bcb16-kube-api-access-d5l25\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831690 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szf2b\" (UniqueName: \"kubernetes.io/projected/f1896cb4-a0d8-4521-83b7-26400e9f186c-kube-api-access-szf2b\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831718 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-run-netns\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-var-lib-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831767 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba81a4c9-7047-466e-b774-30b32290ad01-etc-tuned\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b6e3541-7a83-41ea-bdea-f8e2f4787005-tmp-dir\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-system-cni-dir\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831886 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-slash\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.833144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831947 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovnkube-config\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.831971 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/345979f3-c948-4fc6-87e0-b7e7ac3242e5-kubelet-config\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832001 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npfj4\" (UniqueName: \"kubernetes.io/projected/1f00d08c-a113-479f-a8d3-b8ef63144cc0-kube-api-access-npfj4\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832080 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-kubelet\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-multus-certs\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832142 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-run-ovn-kubernetes\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832183 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-cni-bin\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832210 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-sys-fs\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832232 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-kubelet\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832230 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-var-lib-kubelet\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-log-socket\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832283 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f00d08c-a113-479f-a8d3-b8ef63144cc0-sys-fs\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.834012 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.832321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/601e07b5-4e8e-4935-8288-0be80cac7a3b-host-run-multus-certs\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.834727 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.834709 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba81a4c9-7047-466e-b774-30b32290ad01-tmp\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.834828 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.834768 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba81a4c9-7047-466e-b774-30b32290ad01-etc-tuned\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.835776 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.835757 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3d44367-adf6-4045-b571-8936e08e952b-agent-certs\") pod \"konnectivity-agent-4j88l\" (UID: \"f3d44367-adf6-4045-b571-8936e08e952b\") " pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:48.837475 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.837453 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:28:48.837475 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.837475 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:28:48.837630 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.837494 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:48.837630 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.837562 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:49.337546599 +0000 UTC m=+3.031889567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:48.839629 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.839575 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68552\" (UniqueName: \"kubernetes.io/projected/ba81a4c9-7047-466e-b774-30b32290ad01-kube-api-access-68552\") pod \"tuned-4mhlm\" (UID: \"ba81a4c9-7047-466e-b774-30b32290ad01\") " pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:48.840293 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.840274 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvzn5\" (UniqueName: \"kubernetes.io/projected/601e07b5-4e8e-4935-8288-0be80cac7a3b-kube-api-access-bvzn5\") pod \"multus-jlzjg\" (UID: \"601e07b5-4e8e-4935-8288-0be80cac7a3b\") " pod="openshift-multus/multus-jlzjg" Jun 09 05:28:48.840841 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.840816 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npfj4\" (UniqueName: \"kubernetes.io/projected/1f00d08c-a113-479f-a8d3-b8ef63144cc0-kube-api-access-npfj4\") pod \"aws-ebs-csi-driver-node-mffpr\" (UID: \"1f00d08c-a113-479f-a8d3-b8ef63144cc0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:48.841752 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.841731 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z7xh\" (UniqueName: \"kubernetes.io/projected/879b1a42-fd63-4e8a-be9b-7886e17a8408-kube-api-access-4z7xh\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:48.933109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933039 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szf2b\" (UniqueName: \"kubernetes.io/projected/f1896cb4-a0d8-4521-83b7-26400e9f186c-kube-api-access-szf2b\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.933109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933076 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-run-netns\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933110 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-var-lib-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933138 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b6e3541-7a83-41ea-bdea-f8e2f4787005-tmp-dir\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933162 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-system-cni-dir\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933174 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-run-netns\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933183 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-slash\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933236 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-slash\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933246 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-var-lib-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933278 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-system-cni-dir\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933349 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovnkube-config\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933399 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/345979f3-c948-4fc6-87e0-b7e7ac3242e5-kubelet-config\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933407 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933428 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-run-ovn-kubernetes\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933454 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-cni-bin\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933464 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/345979f3-c948-4fc6-87e0-b7e7ac3242e5-kubelet-config\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-kubelet\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-log-socket\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933494 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-cni-bin\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933513 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-run-ovn-kubernetes\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933522 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovn-node-metrics-cert\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933533 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-log-socket\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933557 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-kubelet\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933566 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsrjd\" (UniqueName: \"kubernetes.io/projected/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-kube-api-access-dsrjd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.933767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933586 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b6e3541-7a83-41ea-bdea-f8e2f4787005-tmp-dir\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933606 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.933951 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-systemd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934025 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqlbt\" (UniqueName: \"kubernetes.io/projected/1560f09c-5460-43eb-b87e-a22e744e1cd4-kube-api-access-gqlbt\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5af69f3-c1ad-4531-b392-4313706bcb16-serviceca\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkdc5\" (UniqueName: \"kubernetes.io/projected/1b6e3541-7a83-41ea-bdea-f8e2f4787005-kube-api-access-vkdc5\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934092 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-etc-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934111 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-ovn\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934132 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5af69f3-c1ad-4531-b392-4313706bcb16-host\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934148 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1560f09c-5460-43eb-b87e-a22e744e1cd4-iptables-alerter-script\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-node-log\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934187 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovnkube-script-lib\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934208 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934233 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934265 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1560f09c-5460-43eb-b87e-a22e744e1cd4-host-slash\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934294 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5af69f3-c1ad-4531-b392-4313706bcb16-host\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.934441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934350 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934369 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934375 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovnkube-config\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934445 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-node-log\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934669 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-etc-openvswitch\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934730 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1560f09c-5460-43eb-b87e-a22e744e1cd4-host-slash\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934741 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.934797 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-ovn\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935094 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovnkube-script-lib\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935134 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935183 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/345979f3-c948-4fc6-87e0-b7e7ac3242e5-dbus\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935197 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935234 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-systemd-units\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-env-overrides\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935289 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/345979f3-c948-4fc6-87e0-b7e7ac3242e5-dbus\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935310 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-cnibin\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935327 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-run-systemd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-cni-netd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935372 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1b6e3541-7a83-41ea-bdea-f8e2f4787005-hosts-file\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935406 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935419 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-os-release\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935507 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-cnibin\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935527 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1b6e3541-7a83-41ea-bdea-f8e2f4787005-hosts-file\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.935650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935561 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-systemd-units\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935810 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5af69f3-c1ad-4531-b392-4313706bcb16-serviceca\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.935838 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:48.935914 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:49.435896103 +0000 UTC m=+3.130239059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935947 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-env-overrides\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.935987 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1896cb4-a0d8-4521-83b7-26400e9f186c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.936040 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1896cb4-a0d8-4521-83b7-26400e9f186c-os-release\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.936074 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-host-cni-netd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.936162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.936084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5l25\" (UniqueName: \"kubernetes.io/projected/a5af69f3-c1ad-4531-b392-4313706bcb16-kube-api-access-d5l25\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.936778 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.936742 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-ovn-node-metrics-cert\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:48.938770 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.938743 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1560f09c-5460-43eb-b87e-a22e744e1cd4-iptables-alerter-script\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.943482 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.943460 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szf2b\" (UniqueName: \"kubernetes.io/projected/f1896cb4-a0d8-4521-83b7-26400e9f186c-kube-api-access-szf2b\") pod \"multus-additional-cni-plugins-mcm2j\" (UID: \"f1896cb4-a0d8-4521-83b7-26400e9f186c\") " pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:48.947495 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.947471 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkdc5\" (UniqueName: \"kubernetes.io/projected/1b6e3541-7a83-41ea-bdea-f8e2f4787005-kube-api-access-vkdc5\") pod \"node-resolver-l6qxp\" (UID: \"1b6e3541-7a83-41ea-bdea-f8e2f4787005\") " pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:48.947673 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.947656 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5l25\" (UniqueName: \"kubernetes.io/projected/a5af69f3-c1ad-4531-b392-4313706bcb16-kube-api-access-d5l25\") pod \"node-ca-vqxk4\" (UID: \"a5af69f3-c1ad-4531-b392-4313706bcb16\") " pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:48.948014 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.947995 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqlbt\" (UniqueName: \"kubernetes.io/projected/1560f09c-5460-43eb-b87e-a22e744e1cd4-kube-api-access-gqlbt\") pod \"iptables-alerter-zcpl5\" (UID: \"1560f09c-5460-43eb-b87e-a22e744e1cd4\") " pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:48.948267 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:48.948248 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsrjd\" (UniqueName: \"kubernetes.io/projected/84f4ae1c-7460-4272-a8ed-ca8cc207ac56-kube-api-access-dsrjd\") pod \"ovnkube-node-jspp7\" (UID: \"84f4ae1c-7460-4272-a8ed-ca8cc207ac56\") " pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:49.024646 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.024625 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" Jun 09 05:28:49.032347 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.032326 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" Jun 09 05:28:49.033445 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.033420 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba81a4c9_7047_466e_b774_30b32290ad01.slice/crio-312b9dc234446b9dc90d91892d3bab59fd8fb5cf45ed5ea2aabc5328a5122885 WatchSource:0}: Error finding container 312b9dc234446b9dc90d91892d3bab59fd8fb5cf45ed5ea2aabc5328a5122885: Status 404 returned error can't find the container with id 312b9dc234446b9dc90d91892d3bab59fd8fb5cf45ed5ea2aabc5328a5122885 Jun 09 05:28:49.040748 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.040721 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f00d08c_a113_479f_a8d3_b8ef63144cc0.slice/crio-a768c8d30bfeb67a4ee743633d9a5aa133ce98b5321990fe94eb06c4a5909332 WatchSource:0}: Error finding container a768c8d30bfeb67a4ee743633d9a5aa133ce98b5321990fe94eb06c4a5909332: Status 404 returned error can't find the container with id a768c8d30bfeb67a4ee743633d9a5aa133ce98b5321990fe94eb06c4a5909332 Jun 09 05:28:49.041698 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.041683 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:28:49.049750 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.049723 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jlzjg" Jun 09 05:28:49.052361 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.052334 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d44367_adf6_4045_b571_8936e08e952b.slice/crio-3b14e70daeb7fd1f273012c764047f2e770c4f9d10c1ddb556c82ef0ca998e25 WatchSource:0}: Error finding container 3b14e70daeb7fd1f273012c764047f2e770c4f9d10c1ddb556c82ef0ca998e25: Status 404 returned error can't find the container with id 3b14e70daeb7fd1f273012c764047f2e770c4f9d10c1ddb556c82ef0ca998e25 Jun 09 05:28:49.059089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.058938 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l6qxp" Jun 09 05:28:49.061608 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.061408 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601e07b5_4e8e_4935_8288_0be80cac7a3b.slice/crio-a66ce1386406365b8199de838d6b79cc311c5cf8352d44b54b18eed1741e7692 WatchSource:0}: Error finding container a66ce1386406365b8199de838d6b79cc311c5cf8352d44b54b18eed1741e7692: Status 404 returned error can't find the container with id a66ce1386406365b8199de838d6b79cc311c5cf8352d44b54b18eed1741e7692 Jun 09 05:28:49.063600 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.063584 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vqxk4" Jun 09 05:28:49.069100 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.069078 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6e3541_7a83_41ea_bdea_f8e2f4787005.slice/crio-a58fc7ca865d123461c7c54a443d67f5c999d69defa6767d5d0f3e7501a6b698 WatchSource:0}: Error finding container a58fc7ca865d123461c7c54a443d67f5c999d69defa6767d5d0f3e7501a6b698: Status 404 returned error can't find the container with id a58fc7ca865d123461c7c54a443d67f5c999d69defa6767d5d0f3e7501a6b698 Jun 09 05:28:49.070788 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.070773 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" Jun 09 05:28:49.073261 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.073237 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5af69f3_c1ad_4531_b392_4313706bcb16.slice/crio-11c395a0e4bda1f7491eaf951cd0f8e6442f3a4ba128d7e3b214b29e36145073 WatchSource:0}: Error finding container 11c395a0e4bda1f7491eaf951cd0f8e6442f3a4ba128d7e3b214b29e36145073: Status 404 returned error can't find the container with id 11c395a0e4bda1f7491eaf951cd0f8e6442f3a4ba128d7e3b214b29e36145073 Jun 09 05:28:49.079953 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.079937 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zcpl5" Jun 09 05:28:49.080223 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.080205 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1896cb4_a0d8_4521_83b7_26400e9f186c.slice/crio-c54fd67abca4c5206777762a6caff212488b68f6726faab545689232373750e2 WatchSource:0}: Error finding container c54fd67abca4c5206777762a6caff212488b68f6726faab545689232373750e2: Status 404 returned error can't find the container with id c54fd67abca4c5206777762a6caff212488b68f6726faab545689232373750e2 Jun 09 05:28:49.086876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.086856 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:28:49.089018 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.088996 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1560f09c_5460_43eb_b87e_a22e744e1cd4.slice/crio-c67c20721b15e672c3c71e2253eb4b149a7e0e544e52a8b7b65e345b59a81bcc WatchSource:0}: Error finding container c67c20721b15e672c3c71e2253eb4b149a7e0e544e52a8b7b65e345b59a81bcc: Status 404 returned error can't find the container with id c67c20721b15e672c3c71e2253eb4b149a7e0e544e52a8b7b65e345b59a81bcc Jun 09 05:28:49.095067 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:28:49.095049 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f4ae1c_7460_4272_a8ed_ca8cc207ac56.slice/crio-e06e656f63c685c918e9a4da4f2390e91b11d4fd55639bbd725065eaae000a65 WatchSource:0}: Error finding container e06e656f63c685c918e9a4da4f2390e91b11d4fd55639bbd725065eaae000a65: Status 404 returned error can't find the container with id e06e656f63c685c918e9a4da4f2390e91b11d4fd55639bbd725065eaae000a65 Jun 09 05:28:49.169881 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.169856 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 05:28:49.339111 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.338940 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:49.339209 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.339115 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:28:49.339209 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.339138 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:28:49.339209 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.339150 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:49.339209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.339177 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:49.339209 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.339191 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:50.339176827 +0000 UTC m=+4.033519784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:49.339444 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.339259 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:49.339444 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.339303 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:50.339287743 +0000 UTC m=+4.033630704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:49.439877 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.439836 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:49.440036 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.440014 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:49.440105 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.440095 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:50.440074584 +0000 UTC m=+4.134417549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:49.792413 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.792336 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 05:23:48 +0000 UTC" deadline="2027-12-05 14:16:09.036297659 +0000 UTC" Jun 09 05:28:49.792413 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.792412 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13064h47m19.243889461s" Jun 09 05:28:49.794224 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.793718 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:49.794224 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:49.793850 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:28:49.806474 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.806414 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zcpl5" event={"ID":"1560f09c-5460-43eb-b87e-a22e744e1cd4","Type":"ContainerStarted","Data":"c67c20721b15e672c3c71e2253eb4b149a7e0e544e52a8b7b65e345b59a81bcc"} Jun 09 05:28:49.809620 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.809567 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerStarted","Data":"c54fd67abca4c5206777762a6caff212488b68f6726faab545689232373750e2"} Jun 09 05:28:49.813005 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.812979 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l6qxp" event={"ID":"1b6e3541-7a83-41ea-bdea-f8e2f4787005","Type":"ContainerStarted","Data":"a58fc7ca865d123461c7c54a443d67f5c999d69defa6767d5d0f3e7501a6b698"} Jun 09 05:28:49.815415 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.815335 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4j88l" event={"ID":"f3d44367-adf6-4045-b571-8936e08e952b","Type":"ContainerStarted","Data":"3b14e70daeb7fd1f273012c764047f2e770c4f9d10c1ddb556c82ef0ca998e25"} Jun 09 05:28:49.817504 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.817473 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" event={"ID":"ba81a4c9-7047-466e-b774-30b32290ad01","Type":"ContainerStarted","Data":"312b9dc234446b9dc90d91892d3bab59fd8fb5cf45ed5ea2aabc5328a5122885"} Jun 09 05:28:49.824523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.823691 2567 generic.go:356] "Generic (PLEG): container finished" podID="8fdafb58a07221d574c211454336b738" containerID="9fc992e2eaa3d7f9eb9bf230694901d0bac2b91e3ab466a8f0e1d82f18c586bc" exitCode=0 Jun 09 05:28:49.824523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.823768 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" event={"ID":"8fdafb58a07221d574c211454336b738","Type":"ContainerDied","Data":"9fc992e2eaa3d7f9eb9bf230694901d0bac2b91e3ab466a8f0e1d82f18c586bc"} Jun 09 05:28:49.836302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.836238 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vqxk4" event={"ID":"a5af69f3-c1ad-4531-b392-4313706bcb16","Type":"ContainerStarted","Data":"11c395a0e4bda1f7491eaf951cd0f8e6442f3a4ba128d7e3b214b29e36145073"} Jun 09 05:28:49.847795 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.847732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jlzjg" event={"ID":"601e07b5-4e8e-4935-8288-0be80cac7a3b","Type":"ContainerStarted","Data":"a66ce1386406365b8199de838d6b79cc311c5cf8352d44b54b18eed1741e7692"} Jun 09 05:28:49.850939 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.850862 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" event={"ID":"1f00d08c-a113-479f-a8d3-b8ef63144cc0","Type":"ContainerStarted","Data":"a768c8d30bfeb67a4ee743633d9a5aa133ce98b5321990fe94eb06c4a5909332"} Jun 09 05:28:49.853199 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:49.853111 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"e06e656f63c685c918e9a4da4f2390e91b11d4fd55639bbd725065eaae000a65"} Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.347407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.347486 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.347606 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.347667 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:52.347648873 +0000 UTC m=+6.041991833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.348086 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.348103 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.348125 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:50.348208 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.348172 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:52.348156645 +0000 UTC m=+6.042499607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:50.449201 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.448586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:50.449201 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.448776 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:50.449201 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.448840 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:52.448820853 +0000 UTC m=+6.143163817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:50.795424 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.795337 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:50.795862 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.795519 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:28:50.795929 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.795895 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:50.796049 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:50.795981 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:28:50.796187 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.796143 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 05:23:48 +0000 UTC" deadline="2028-01-11 00:18:59.002867507 +0000 UTC" Jun 09 05:28:50.796244 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.796189 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13938h50m8.206682163s" Jun 09 05:28:50.876460 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.876166 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" event={"ID":"7bbdc86751d54a40c4e45022ba51be3b","Type":"ContainerStarted","Data":"7fa8fde02f668f02361cc38ba4a7179a4d0c19fec89362d388e8fc3f4d66e6e1"} Jun 09 05:28:50.880526 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.880405 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" event={"ID":"8fdafb58a07221d574c211454336b738","Type":"ContainerStarted","Data":"76d0348e3cc10e3d44408f16fd4eebebe5b96e894941c33d5424b5b1712f6873"} Jun 09 05:28:50.903514 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:50.903004 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-121.ec2.internal" podStartSLOduration=3.902983974 podStartE2EDuration="3.902983974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:28:50.889297819 +0000 UTC m=+4.583640799" watchObservedRunningTime="2026-06-09 05:28:50.902983974 +0000 UTC m=+4.597326958" Jun 09 05:28:51.793734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:51.793701 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:51.793928 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:51.793899 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:28:52.367459 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:52.367416 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:52.367512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.367593 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.367625 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.367638 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.367668 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.367697 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:56.367677188 +0000 UTC m=+10.062020168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:52.367886 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.367720 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:56.36770416 +0000 UTC m=+10.062047124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:52.468311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:52.468226 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:52.468487 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.468401 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:52.468487 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.468470 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:28:56.468449516 +0000 UTC m=+10.162792475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:52.795346 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:52.795264 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:52.795533 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.795421 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:28:52.795533 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:52.795441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:52.795533 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:52.795520 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:28:53.794032 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:53.793547 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:53.794032 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:53.793695 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:28:54.794000 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:54.793782 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:54.794000 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:54.793925 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:28:54.794000 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:54.793969 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:54.794582 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:54.794088 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:28:55.793340 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:55.793002 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:55.793340 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:55.793154 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:28:56.401312 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:56.401272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:56.401351 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.401453 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.401476 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.401488 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.401492 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.401566 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:04.401546348 +0000 UTC m=+18.095889330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:28:56.401735 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.401584 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:04.401576932 +0000 UTC m=+18.095919889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:28:56.501853 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:56.501816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:56.502010 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.501968 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:56.502066 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.502044 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:04.502023493 +0000 UTC m=+18.196366453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:28:56.794466 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:56.794380 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:56.794622 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:56.794483 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:56.794622 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.794507 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:28:56.794746 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:56.794621 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:28:57.792912 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:57.792876 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:57.793373 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:57.793012 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:28:58.793258 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:58.793227 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:28:58.793717 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:58.793350 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:28:58.793717 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:58.793413 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:28:58.793717 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:58.793549 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:28:59.793757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:28:59.793722 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:28:59.794176 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:28:59.793846 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:00.793215 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:00.793171 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:00.793416 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:00.793224 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:00.793416 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:00.793318 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:00.793543 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:00.793442 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:01.793032 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:01.793003 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:01.793475 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:01.793118 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:02.793913 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:02.793875 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:02.794360 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:02.793922 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:02.794360 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:02.794006 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:02.794360 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:02.794147 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:03.793685 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:03.793650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:03.793849 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:03.793777 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:04.460534 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:04.460498 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:04.460564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.460661 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.460666 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.460684 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.460696 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.460725 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:20.460705718 +0000 UTC m=+34.155048680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:29:04.460961 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.460740 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:20.460733933 +0000 UTC m=+34.155076889 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:29:04.561253 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:04.561220 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:04.561421 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.561376 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:29:04.561466 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.561451 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:20.561433242 +0000 UTC m=+34.255776202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:29:04.793651 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:04.793577 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:04.793799 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.793691 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:04.793799 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:04.793751 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:04.793923 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:04.793900 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:05.793876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:05.793843 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:05.794279 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:05.793969 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:06.793734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:06.793698 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:06.793934 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:06.793822 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:06.793934 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:06.793842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:06.794349 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:06.793939 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:07.793729 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:07.793691 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:07.794006 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:07.793805 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:08.793769 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.793563 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:08.793929 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.793616 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:08.793929 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:08.793874 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:08.793929 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:08.793916 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:08.919374 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.919339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l6qxp" event={"ID":"1b6e3541-7a83-41ea-bdea-f8e2f4787005","Type":"ContainerStarted","Data":"a45535d09f7116103ef063abad527c821e2491ec094ff21d95e716a49b8310c3"} Jun 09 05:29:08.920630 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.920598 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4j88l" event={"ID":"f3d44367-adf6-4045-b571-8936e08e952b","Type":"ContainerStarted","Data":"065441875dcea26fff04ddc63e4c0b92bb831eae1717f22961c250a25404566e"} Jun 09 05:29:08.921771 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.921744 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" event={"ID":"ba81a4c9-7047-466e-b774-30b32290ad01","Type":"ContainerStarted","Data":"012c2926d2476dfc6d5f80723f9b20c05a627904aa8751995834e4e09cbede4d"} Jun 09 05:29:08.923126 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.923098 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vqxk4" event={"ID":"a5af69f3-c1ad-4531-b392-4313706bcb16","Type":"ContainerStarted","Data":"0f9e253524ea84c6a06695af635b21d0953d64c18868f7936568f0ed7bb8d406"} Jun 09 05:29:08.924540 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.924519 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jlzjg" event={"ID":"601e07b5-4e8e-4935-8288-0be80cac7a3b","Type":"ContainerStarted","Data":"b08feeea4bd0aecd36551bc2fcab4b5d86be5a04b83da53bf00c90862a757f86"} Jun 09 05:29:08.925909 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.925886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" event={"ID":"1f00d08c-a113-479f-a8d3-b8ef63144cc0","Type":"ContainerStarted","Data":"678518bb4a773d00fc35cb025c7ad20a891216c3c455721f48c7f2de3babd3c3"} Jun 09 05:29:08.927261 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.927241 2567 generic.go:356] "Generic (PLEG): container finished" podID="f1896cb4-a0d8-4521-83b7-26400e9f186c" containerID="6865ff377f1025b2fd9c931c56c69591fe56646eba6b36bffc4f381f76fcec4c" exitCode=0 Jun 09 05:29:08.927335 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.927267 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerDied","Data":"6865ff377f1025b2fd9c931c56c69591fe56646eba6b36bffc4f381f76fcec4c"} Jun 09 05:29:08.936232 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.936187 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-121.ec2.internal" podStartSLOduration=21.936172055 podStartE2EDuration="21.936172055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:28:50.903489986 +0000 UTC m=+4.597832966" watchObservedRunningTime="2026-06-09 05:29:08.936172055 +0000 UTC m=+22.630515037" Jun 09 05:29:08.936507 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.936477 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l6qxp" podStartSLOduration=3.085485083 podStartE2EDuration="21.936468273s" totalImagesPullingTime="18.85098319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:08.936301328 +0000 UTC m=+22.630644289" watchObservedRunningTime="2026-06-09 05:29:08.936468273 +0000 UTC m=+22.630811253" Jun 09 05:29:08.962010 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.961977 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4j88l" podStartSLOduration=7.611255658 podStartE2EDuration="21.961965191s" totalImagesPullingTime="14.350709533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:08.949074857 +0000 UTC m=+22.643417837" watchObservedRunningTime="2026-06-09 05:29:08.961965191 +0000 UTC m=+22.656308169" Jun 09 05:29:08.962108 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:08.962065 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vqxk4" podStartSLOduration=3.107373394 podStartE2EDuration="21.962060757s" totalImagesPullingTime="18.854687363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:08.961620882 +0000 UTC m=+22.655963863" watchObservedRunningTime="2026-06-09 05:29:08.962060757 +0000 UTC m=+22.656403735" Jun 09 05:29:09.002229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.002185 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jlzjg" podStartSLOduration=2.915109346 podStartE2EDuration="22.002170838s" totalImagesPullingTime="19.087061492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:09.001830947 +0000 UTC m=+22.696173926" watchObservedRunningTime="2026-06-09 05:29:09.002170838 +0000 UTC m=+22.696513816" Jun 09 05:29:09.018603 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.018568 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4mhlm" podStartSLOduration=4.120406771 podStartE2EDuration="23.018555663s" totalImagesPullingTime="18.898148892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:09.018309168 +0000 UTC m=+22.712652146" watchObservedRunningTime="2026-06-09 05:29:09.018555663 +0000 UTC m=+22.712898641" Jun 09 05:29:09.414858 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.414661 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 05:29:09.792779 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.792646 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T05:29:09.414855868Z","UUID":"b2b1b51f-7058-4cf9-bcbb-28f610657e24","Handler":null,"Name":"","Endpoint":""} Jun 09 05:29:09.792966 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.792927 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:09.793058 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:09.793039 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:09.794237 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.794205 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 05:29:09.794918 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.794244 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 05:29:09.939741 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.939704 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"16df2a7d0b181757eba7426b4ed28ef46a357cd73dad917d626eafb04cd80065"} Jun 09 05:29:09.939911 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.939830 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"bd7aa2a843dc7e671640c0bd06b54b9fa214f7943c27f792a1624b817244d78f"} Jun 09 05:29:09.939911 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.939884 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"06bfe11384a68d21f5c82ebcf4111a0475c7f7e4bbe7b5f81d7b109798cd9aec"} Jun 09 05:29:09.940023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.939932 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"1e1139baab1d718557d13e1df9328bb752cdecdfc0d3a62f038da1d8289ea6ed"} Jun 09 05:29:09.940023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.939980 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"28d99716b90267f603df86535507c47bd80d5a75cb8c572ce93d41b608a3b09a"} Jun 09 05:29:09.940127 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.940025 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"eb3f8a43ddd5c8c68b6757e95b453cb66c75910749443b029a7b679cde99a21a"} Jun 09 05:29:09.941655 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.941627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zcpl5" event={"ID":"1560f09c-5460-43eb-b87e-a22e744e1cd4","Type":"ContainerStarted","Data":"3c473d84c12ca47640521f74644c475598388460aa787bcad06d709237ab5a13"} Jun 09 05:29:09.944399 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.944082 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" event={"ID":"1f00d08c-a113-479f-a8d3-b8ef63144cc0","Type":"ContainerStarted","Data":"989be278ae495b3908ad74cdfaab04f4837f7c6b2e99364609cbd1ea30c7366b"} Jun 09 05:29:09.955481 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:09.955437 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zcpl5" podStartSLOduration=4.116032392 podStartE2EDuration="22.955419595s" totalImagesPullingTime="18.839387203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:09.955102267 +0000 UTC m=+23.649445247" watchObservedRunningTime="2026-06-09 05:29:09.955419595 +0000 UTC m=+23.649762575" Jun 09 05:29:10.793954 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:10.793923 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:10.794310 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:10.793929 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:10.794310 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:10.794026 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:10.794310 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:10.794135 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:11.793407 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:11.793194 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:11.793617 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:11.793530 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:11.951717 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:11.951615 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" event={"ID":"1f00d08c-a113-479f-a8d3-b8ef63144cc0","Type":"ContainerStarted","Data":"401632410b656bba0226401fdbf2e6eb63fa2a272d454889e1517809cb1a7b75"} Jun 09 05:29:12.581241 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.581132 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:29:12.581852 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.581830 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:29:12.609517 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.609463 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mffpr" podStartSLOduration=5.120450616 podStartE2EDuration="26.609445563s" totalImagesPullingTime="21.488994947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:11.978934976 +0000 UTC m=+25.673277965" watchObservedRunningTime="2026-06-09 05:29:12.609445563 +0000 UTC m=+26.303788543" Jun 09 05:29:12.796896 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.796871 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:12.796896 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.796880 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:12.797129 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:12.796967 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:12.797129 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:12.797110 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:12.953993 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.953963 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:29:12.954649 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:12.954633 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4j88l" Jun 09 05:29:13.793334 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:13.793296 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:13.793529 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:13.793445 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:13.958733 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:13.958700 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"a728d100b98bf1d2e7c29ab321670c1bfcf19b9fd60e88963d490a3c819e760f"} Jun 09 05:29:13.960437 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:13.960415 2567 generic.go:356] "Generic (PLEG): container finished" podID="f1896cb4-a0d8-4521-83b7-26400e9f186c" containerID="c4ab44245fe5a7458ccb3688d4f588a025e8970648de7bc4fc31f8538c4d268d" exitCode=0 Jun 09 05:29:13.960542 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:13.960502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerDied","Data":"c4ab44245fe5a7458ccb3688d4f588a025e8970648de7bc4fc31f8538c4d268d"} Jun 09 05:29:14.796674 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:14.796643 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:14.796829 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:14.796747 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:14.796829 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:14.796789 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:14.796904 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:14.796866 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:15.793420 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.793216 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:15.793873 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:15.793501 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:15.970321 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.970286 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" event={"ID":"84f4ae1c-7460-4272-a8ed-ca8cc207ac56","Type":"ContainerStarted","Data":"31725e67b167daca458700244e128af46935b6bdc246670f4cf1d3f68ed65776"} Jun 09 05:29:15.970667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.970494 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:29:15.970667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.970577 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:29:15.970667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.970613 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:29:15.972586 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.972559 2567 generic.go:356] "Generic (PLEG): container finished" podID="f1896cb4-a0d8-4521-83b7-26400e9f186c" containerID="8d45ae1237daa79294d2f978f643a8fb09579d4de3ebc6f0939746785a3f3a28" exitCode=0 Jun 09 05:29:15.972692 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.972613 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerDied","Data":"8d45ae1237daa79294d2f978f643a8fb09579d4de3ebc6f0939746785a3f3a28"} Jun 09 05:29:15.986162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.986140 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:29:15.986244 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.986235 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:29:15.994415 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:15.994360 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" podStartSLOduration=9.18544519 podStartE2EDuration="28.994349045s" totalImagesPullingTime="19.808903855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:15.993068393 +0000 UTC m=+29.687411371" watchObservedRunningTime="2026-06-09 05:29:15.994349045 +0000 UTC m=+29.688692058" Jun 09 05:29:16.795852 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:16.795823 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:16.795852 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:16.795850 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:16.796330 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:16.795947 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:16.796330 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:16.796048 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:17.269229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.269196 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kwjn"] Jun 09 05:29:17.269406 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.269334 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:17.269485 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:17.269463 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:17.272006 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.271982 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wfj72"] Jun 09 05:29:17.272107 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.272082 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:17.272181 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:17.272158 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:17.283081 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.283055 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-28hwr"] Jun 09 05:29:17.283196 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.283155 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:17.283268 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:17.283247 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:17.979125 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.978962 2567 generic.go:356] "Generic (PLEG): container finished" podID="f1896cb4-a0d8-4521-83b7-26400e9f186c" containerID="dc5b4ddabf51b0b4da362d451fbd78724027b8d9baf4400b42fe09bf98f64ec1" exitCode=0 Jun 09 05:29:17.979465 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:17.979024 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerDied","Data":"dc5b4ddabf51b0b4da362d451fbd78724027b8d9baf4400b42fe09bf98f64ec1"} Jun 09 05:29:18.793268 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:18.793241 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:18.793268 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:18.793239 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:18.793531 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:18.793406 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:18.793531 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:18.793441 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:18.793608 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:18.793555 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:18.793674 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:18.793655 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:20.480542 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:20.480648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.480782 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.480847 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:52.480826899 +0000 UTC m=+66.175169861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.481046 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.481077 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.481094 2567 projected.go:196] Error preparing data for projected volume kube-api-access-hxt72 for pod openshift-network-diagnostics/network-check-target-6kwjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:29:20.482988 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.481194 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72 podName:75b33e34-eec5-468c-93ef-17dcfa485231 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:52.481162984 +0000 UTC m=+66.175505958 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxt72" (UniqueName: "kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72") pod "network-check-target-6kwjn" (UID: "75b33e34-eec5-468c-93ef-17dcfa485231") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 05:29:20.581541 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:20.581505 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:20.581740 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.581681 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 05:29:20.581801 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.581760 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret podName:345979f3-c948-4fc6-87e0-b7e7ac3242e5 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:52.58173963 +0000 UTC m=+66.276082603 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret") pod "global-pull-secret-syncer-28hwr" (UID: "345979f3-c948-4fc6-87e0-b7e7ac3242e5") : object "kube-system"/"original-pull-secret" not registered Jun 09 05:29:20.797317 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:20.797237 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:20.797479 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:20.797357 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:20.797479 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:20.797365 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:20.797479 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.797365 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-28hwr" podUID="345979f3-c948-4fc6-87e0-b7e7ac3242e5" Jun 09 05:29:20.797633 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.797476 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kwjn" podUID="75b33e34-eec5-468c-93ef-17dcfa485231" Jun 09 05:29:20.797633 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:20.797585 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:29:21.147604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.147572 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-121.ec2.internal" event="NodeReady" Jun 09 05:29:21.147750 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.147715 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 05:29:21.175891 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.175858 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-864f978974-4b5sl"] Jun 09 05:29:21.180718 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.180689 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.183278 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.183217 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2rk2w\"" Jun 09 05:29:21.183278 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.183250 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 09 05:29:21.183466 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.183257 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 09 05:29:21.184031 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.184007 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 09 05:29:21.188325 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.187850 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8r8dx"] Jun 09 05:29:21.190208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.190062 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 09 05:29:21.191655 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.191615 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864f978974-4b5sl"] Jun 09 05:29:21.191655 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.191654 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dhhmg"] Jun 09 05:29:21.191804 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.191749 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.194194 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.194049 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 05:29:21.194194 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.194063 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ww7k9\"" Jun 09 05:29:21.194194 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.194078 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 05:29:21.195542 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.195454 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.197433 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.197414 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 05:29:21.197523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.197472 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n9hnl\"" Jun 09 05:29:21.197605 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.197544 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 05:29:21.197732 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.197714 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 05:29:21.199808 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.199617 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8r8dx"] Jun 09 05:29:21.204999 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.204942 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dhhmg"] Jun 09 05:29:21.285754 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285716 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40108820-3325-4e88-a8b1-056266249a9b-tmp-dir\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.285922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285763 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-bound-sa-token\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.285922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285798 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-image-registry-private-configuration\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.285922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285821 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-trusted-ca\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.285922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285878 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9a9a461-a7dc-4aef-9008-e2418f03da29-cert\") pod \"ingress-canary-dhhmg\" (UID: \"c9a9a461-a7dc-4aef-9008-e2418f03da29\") " pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.285922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285932 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40108820-3325-4e88-a8b1-056266249a9b-config-volume\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285949 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf8xs\" (UniqueName: \"kubernetes.io/projected/40108820-3325-4e88-a8b1-056266249a9b-kube-api-access-xf8xs\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-ca-trust-extracted\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.285987 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-installation-pull-secrets\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.286049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22gr\" (UniqueName: \"kubernetes.io/projected/c9a9a461-a7dc-4aef-9008-e2418f03da29-kube-api-access-k22gr\") pod \"ingress-canary-dhhmg\" (UID: \"c9a9a461-a7dc-4aef-9008-e2418f03da29\") " pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.286087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv4r7\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-kube-api-access-nv4r7\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.286132 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.286184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.286150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-certificates\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387073 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387031 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-image-registry-private-configuration\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387073 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387077 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-trusted-ca\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387109 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9a9a461-a7dc-4aef-9008-e2418f03da29-cert\") pod \"ingress-canary-dhhmg\" (UID: \"c9a9a461-a7dc-4aef-9008-e2418f03da29\") " pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.387305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.387305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387176 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40108820-3325-4e88-a8b1-056266249a9b-config-volume\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.387305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387204 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf8xs\" (UniqueName: \"kubernetes.io/projected/40108820-3325-4e88-a8b1-056266249a9b-kube-api-access-xf8xs\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.387305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-ca-trust-extracted\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-installation-pull-secrets\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387590 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22gr\" (UniqueName: \"kubernetes.io/projected/c9a9a461-a7dc-4aef-9008-e2418f03da29-kube-api-access-k22gr\") pod \"ingress-canary-dhhmg\" (UID: \"c9a9a461-a7dc-4aef-9008-e2418f03da29\") " pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.387590 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387422 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv4r7\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-kube-api-access-nv4r7\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387590 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387590 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387516 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-certificates\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.387590 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40108820-3325-4e88-a8b1-056266249a9b-tmp-dir\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.387800 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.387617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-bound-sa-token\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.387957 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.388028 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:29:21.888006706 +0000 UTC m=+35.582349674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.388357 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40108820-3325-4e88-a8b1-056266249a9b-config-volume\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.388489 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.388507 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.388561 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:21.888545946 +0000 UTC m=+35.582888902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:21.388678 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.388637 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-ca-trust-extracted\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.389102 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.388699 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40108820-3325-4e88-a8b1-056266249a9b-tmp-dir\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.389102 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.388717 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-trusted-ca\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.389102 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.388811 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-certificates\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.392255 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.392234 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-image-registry-private-configuration\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.392357 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.392259 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-installation-pull-secrets\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.392357 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.392265 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9a9a461-a7dc-4aef-9008-e2418f03da29-cert\") pod \"ingress-canary-dhhmg\" (UID: \"c9a9a461-a7dc-4aef-9008-e2418f03da29\") " pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.396131 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.396108 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22gr\" (UniqueName: \"kubernetes.io/projected/c9a9a461-a7dc-4aef-9008-e2418f03da29-kube-api-access-k22gr\") pod \"ingress-canary-dhhmg\" (UID: \"c9a9a461-a7dc-4aef-9008-e2418f03da29\") " pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.396661 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.396637 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv4r7\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-kube-api-access-nv4r7\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.396857 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.396823 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-bound-sa-token\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.397029 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.397010 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf8xs\" (UniqueName: \"kubernetes.io/projected/40108820-3325-4e88-a8b1-056266249a9b-kube-api-access-xf8xs\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.510834 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.510748 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dhhmg" Jun 09 05:29:21.893951 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.892800 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:21.893951 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.893351 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:21.893951 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.893528 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:21.893951 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.893588 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:29:22.893569362 +0000 UTC m=+36.587912335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:21.897203 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.897178 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:21.897203 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.897205 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:21.897326 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:21.897261 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:22.897245278 +0000 UTC m=+36.591588251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:21.941355 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.941323 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dhhmg"] Jun 09 05:29:21.944726 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:29:21.944701 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a9a461_a7dc_4aef_9008_e2418f03da29.slice/crio-db08d47f20d543823380cd0d8746d9ec8110c1a65170093f4ec0b739fe8522f8 WatchSource:0}: Error finding container db08d47f20d543823380cd0d8746d9ec8110c1a65170093f4ec0b739fe8522f8: Status 404 returned error can't find the container with id db08d47f20d543823380cd0d8746d9ec8110c1a65170093f4ec0b739fe8522f8 Jun 09 05:29:21.990368 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.990337 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerStarted","Data":"6dc3f1442ce8099e86bc3ce15ccb6d0b114e042ffa5ee56e66416812f22fb661"} Jun 09 05:29:21.991602 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:21.991577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dhhmg" event={"ID":"c9a9a461-a7dc-4aef-9008-e2418f03da29","Type":"ContainerStarted","Data":"db08d47f20d543823380cd0d8746d9ec8110c1a65170093f4ec0b739fe8522f8"} Jun 09 05:29:22.796184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.796114 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:22.796759 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.796203 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:22.796759 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.796344 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:22.799937 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.799913 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 05:29:22.799937 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.799927 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tg4g2\"" Jun 09 05:29:22.800136 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.799939 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vlvww\"" Jun 09 05:29:22.800136 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.799959 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 05:29:22.800136 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.799990 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 05:29:22.800248 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.800235 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 05:29:22.900991 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.900954 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:22.901161 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.901073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:22.901161 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:22.901098 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:22.901161 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:22.901161 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:29:24.901140179 +0000 UTC m=+38.595483152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:22.901333 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:22.901194 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:22.901333 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:22.901208 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:22.901333 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:22.901274 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:24.901256859 +0000 UTC m=+38.595599823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:22.997319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.997284 2567 generic.go:356] "Generic (PLEG): container finished" podID="f1896cb4-a0d8-4521-83b7-26400e9f186c" containerID="6dc3f1442ce8099e86bc3ce15ccb6d0b114e042ffa5ee56e66416812f22fb661" exitCode=0 Jun 09 05:29:22.997512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:22.997337 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerDied","Data":"6dc3f1442ce8099e86bc3ce15ccb6d0b114e042ffa5ee56e66416812f22fb661"} Jun 09 05:29:24.002851 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:24.002816 2567 generic.go:356] "Generic (PLEG): container finished" podID="f1896cb4-a0d8-4521-83b7-26400e9f186c" containerID="3579e7d09308f46312a2bf750c62e2b2ba6c8804125a2648dc7ee775b5196eb8" exitCode=0 Jun 09 05:29:24.003299 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:24.002882 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerDied","Data":"3579e7d09308f46312a2bf750c62e2b2ba6c8804125a2648dc7ee775b5196eb8"} Jun 09 05:29:24.917107 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:24.916924 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:24.917266 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:24.917140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:24.917266 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:24.917073 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:24.917266 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:24.917186 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:24.917266 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:24.917235 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:28.917219211 +0000 UTC m=+42.611562168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:24.917266 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:24.917246 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:24.917481 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:24.917283 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:29:28.917272462 +0000 UTC m=+42.611615419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:25.008217 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.008185 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" event={"ID":"f1896cb4-a0d8-4521-83b7-26400e9f186c","Type":"ContainerStarted","Data":"a0ae0fbbf7b387cdf0194b11ef745ca95127519e2a8762e20b8778ed4885bb93"} Jun 09 05:29:25.009656 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.009637 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dhhmg" event={"ID":"c9a9a461-a7dc-4aef-9008-e2418f03da29","Type":"ContainerStarted","Data":"d63d3fac52fbdefa2c3890718857db3c316c9ec77d97b2984149b55a7d843093"} Jun 09 05:29:25.030339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.030297 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mcm2j" podStartSLOduration=9.089433306 podStartE2EDuration="38.030285429s" totalImagesPullingTime="28.940852123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:25.028731838 +0000 UTC m=+38.723074817" watchObservedRunningTime="2026-06-09 05:29:25.030285429 +0000 UTC m=+38.724628407" Jun 09 05:29:25.043352 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.043312 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dhhmg" podStartSLOduration=1.461967912 podStartE2EDuration="4.043298009s" totalImagesPullingTime="2.581330097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:25.043090719 +0000 UTC m=+38.737433698" watchObservedRunningTime="2026-06-09 05:29:25.043298009 +0000 UTC m=+38.737640990" Jun 09 05:29:25.082430 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.082398 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:25.160022 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.159989 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:25.208543 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.208482 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:25.256546 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.256521 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:25.754952 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:25.754917 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:28.946488 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:28.946447 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:28.946863 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:28.946521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:28.946863 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:28.946637 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:28.946863 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:28.946647 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:28.946863 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:28.946662 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:28.946863 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:28.946704 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:29:36.946684081 +0000 UTC m=+50.641027054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:28.946863 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:28.946718 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:36.946712405 +0000 UTC m=+50.641055362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:32.733916 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:32.733872 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:32.832121 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:32.832086 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:33.247026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:33.246990 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:33.346990 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:33.346963 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:33.594094 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:33.594009 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:33.975326 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:33.975294 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:34.075655 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:34.075614 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:34.769349 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:34.769314 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:35.170895 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:35.170861 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:35.219904 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:35.219877 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:37.012804 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:37.012766 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:37.013248 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:37.012824 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:37.013248 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:37.012926 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:37.013248 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:37.012951 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:37.013248 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:37.012974 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:37.013248 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:37.012978 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:29:53.012963169 +0000 UTC m=+66.707306126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:37.013248 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:37.013043 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:29:53.013022643 +0000 UTC m=+66.707365607 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:38.633760 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:38.633726 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:47.338288 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:47.338250 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:47.391986 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:47.391953 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:47.637214 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:47.637184 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:47.737994 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:47.737956 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:47.837697 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:47.837663 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:47.989541 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:47.989465 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jspp7" Jun 09 05:29:48.283578 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:48.283486 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:48.332110 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:48.332078 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:48.676798 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:48.676760 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:48.729243 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:48.729197 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:49.111063 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:49.110975 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:49.173961 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:49.173932 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:50.623101 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:50.623069 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:50.669077 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:50.669027 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:29:52.515535 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.515493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:52.515931 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.515581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:29:52.518375 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.518357 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 05:29:52.518446 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.518405 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 05:29:52.526249 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:52.526225 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 05:29:52.526323 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:52.526300 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:30:56.526282696 +0000 UTC m=+130.220625653 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : secret "metrics-daemon-secret" not found Jun 09 05:29:52.528315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.528296 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 05:29:52.541649 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.541615 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxt72\" (UniqueName: \"kubernetes.io/projected/75b33e34-eec5-468c-93ef-17dcfa485231-kube-api-access-hxt72\") pod \"network-check-target-6kwjn\" (UID: \"75b33e34-eec5-468c-93ef-17dcfa485231\") " pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:52.616446 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.616379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:52.618785 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.618761 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 05:29:52.629736 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.629706 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/345979f3-c948-4fc6-87e0-b7e7ac3242e5-original-pull-secret\") pod \"global-pull-secret-syncer-28hwr\" (UID: \"345979f3-c948-4fc6-87e0-b7e7ac3242e5\") " pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:52.807991 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.807916 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vlvww\"" Jun 09 05:29:52.810743 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.810725 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-28hwr" Jun 09 05:29:52.816479 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.816463 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:52.942847 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.942770 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-28hwr"] Jun 09 05:29:52.946515 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:29:52.946489 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345979f3_c948_4fc6_87e0_b7e7ac3242e5.slice/crio-e1fd376c688c4f1a8277fdb7a144e9e1a62af345311b13f64f35b10d5ddb5ea0 WatchSource:0}: Error finding container e1fd376c688c4f1a8277fdb7a144e9e1a62af345311b13f64f35b10d5ddb5ea0: Status 404 returned error can't find the container with id e1fd376c688c4f1a8277fdb7a144e9e1a62af345311b13f64f35b10d5ddb5ea0 Jun 09 05:29:52.957357 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:52.957333 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kwjn"] Jun 09 05:29:52.960090 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:29:52.960066 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b33e34_eec5_468c_93ef_17dcfa485231.slice/crio-315be878df1aa1cc1a9645fd72f9abaa51f42d2a66c8a3499465167bbef7814a WatchSource:0}: Error finding container 315be878df1aa1cc1a9645fd72f9abaa51f42d2a66c8a3499465167bbef7814a: Status 404 returned error can't find the container with id 315be878df1aa1cc1a9645fd72f9abaa51f42d2a66c8a3499465167bbef7814a Jun 09 05:29:53.020512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:53.020471 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:29:53.020694 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:53.020535 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:29:53.020694 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:53.020622 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:29:53.020694 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:53.020637 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:29:53.020694 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:53.020683 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:29:53.020847 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:53.020691 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:30:25.02067666 +0000 UTC m=+98.715019620 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:29:53.020847 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:29:53.020748 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:30:25.020730089 +0000 UTC m=+98.715073059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:29:53.082825 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:53.082731 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kwjn" event={"ID":"75b33e34-eec5-468c-93ef-17dcfa485231","Type":"ContainerStarted","Data":"315be878df1aa1cc1a9645fd72f9abaa51f42d2a66c8a3499465167bbef7814a"} Jun 09 05:29:53.083713 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:53.083686 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-28hwr" event={"ID":"345979f3-c948-4fc6-87e0-b7e7ac3242e5","Type":"ContainerStarted","Data":"e1fd376c688c4f1a8277fdb7a144e9e1a62af345311b13f64f35b10d5ddb5ea0"} Jun 09 05:29:58.098054 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:58.098017 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kwjn" event={"ID":"75b33e34-eec5-468c-93ef-17dcfa485231","Type":"ContainerStarted","Data":"c1c6606317a863bc3c4d89ec490bd053101bd81d4ad6a3d16d686e4c7913f4c5"} Jun 09 05:29:58.098577 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:58.098114 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:29:58.099432 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:58.099410 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-28hwr" event={"ID":"345979f3-c948-4fc6-87e0-b7e7ac3242e5","Type":"ContainerStarted","Data":"3b91abb91b70c11327b807302427f18f48983ff42f88554c2c84f9b05d00d33f"} Jun 09 05:29:58.112026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:58.111976 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6kwjn" podStartSLOduration=67.524418405 podStartE2EDuration="1m12.111962894s" totalImagesPullingTime="4.587544489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:58.111167548 +0000 UTC m=+71.805510539" watchObservedRunningTime="2026-06-09 05:29:58.111962894 +0000 UTC m=+71.806305874" Jun 09 05:29:58.123652 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:29:58.123607 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-28hwr" podStartSLOduration=66.519343041 podStartE2EDuration="1m11.123593734s" totalImagesPullingTime="4.604250693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:29:58.122993171 +0000 UTC m=+71.817336154" watchObservedRunningTime="2026-06-09 05:29:58.123593734 +0000 UTC m=+71.817936712" Jun 09 05:30:05.559787 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.559749 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:05.659372 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.659338 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:05.964523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.964493 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk"] Jun 09 05:30:05.967084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.967068 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:05.969374 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.969351 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 09 05:30:05.969514 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.969417 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 09 05:30:05.970305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.970287 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 09 05:30:05.970360 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.970301 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 09 05:30:05.976423 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:05.976400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk"] Jun 09 05:30:06.014002 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.013961 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5669941-51c5-4cc6-a407-42660f34eda1-tmp\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.014175 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.014030 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bdsm\" (UniqueName: \"kubernetes.io/projected/f5669941-51c5-4cc6-a407-42660f34eda1-kube-api-access-7bdsm\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.014175 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.014067 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f5669941-51c5-4cc6-a407-42660f34eda1-klusterlet-config\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.114395 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.114363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bdsm\" (UniqueName: \"kubernetes.io/projected/f5669941-51c5-4cc6-a407-42660f34eda1-kube-api-access-7bdsm\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.114475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.114423 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f5669941-51c5-4cc6-a407-42660f34eda1-klusterlet-config\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.114475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.114457 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5669941-51c5-4cc6-a407-42660f34eda1-tmp\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.114860 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.114844 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5669941-51c5-4cc6-a407-42660f34eda1-tmp\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.116963 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.116939 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f5669941-51c5-4cc6-a407-42660f34eda1-klusterlet-config\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.121965 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.121943 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bdsm\" (UniqueName: \"kubernetes.io/projected/f5669941-51c5-4cc6-a407-42660f34eda1-kube-api-access-7bdsm\") pod \"klusterlet-addon-workmgr-9bdd66bf8-22jtk\" (UID: \"f5669941-51c5-4cc6-a407-42660f34eda1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.277018 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.276848 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:06.394835 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:06.394801 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk"] Jun 09 05:30:06.398118 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:30:06.398090 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5669941_51c5_4cc6_a407_42660f34eda1.slice/crio-bebe015aff96378e9efacdbebb3ff63dff7136f923a3c3ce5f74975b0df920e5 WatchSource:0}: Error finding container bebe015aff96378e9efacdbebb3ff63dff7136f923a3c3ce5f74975b0df920e5: Status 404 returned error can't find the container with id bebe015aff96378e9efacdbebb3ff63dff7136f923a3c3ce5f74975b0df920e5 Jun 09 05:30:07.121551 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:07.121514 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" event={"ID":"f5669941-51c5-4cc6-a407-42660f34eda1","Type":"ContainerStarted","Data":"bebe015aff96378e9efacdbebb3ff63dff7136f923a3c3ce5f74975b0df920e5"} Jun 09 05:30:10.327311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:10.327269 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:10.378516 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:10.378477 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:10.626512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:10.626475 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:10.728039 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:10.728009 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:10.776809 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:10.776771 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:30:11.132895 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:11.132858 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" event={"ID":"f5669941-51c5-4cc6-a407-42660f34eda1","Type":"ContainerStarted","Data":"cbe8f75515da7f9da47f99a90ef70023c845f0c895f19084ce7aed7f9fe0d076"} Jun 09 05:30:11.133092 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:11.133073 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:11.134798 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:11.134771 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:30:11.147714 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:11.147659 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" podStartSLOduration=1.982783554 podStartE2EDuration="6.147644891s" totalImagesPullingTime="4.164861337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:30:11.147185152 +0000 UTC m=+84.841528130" watchObservedRunningTime="2026-06-09 05:30:11.147644891 +0000 UTC m=+84.841987914" Jun 09 05:30:25.055905 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:25.055849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:30:25.056302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:25.055927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:30:25.056302 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:25.056017 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:30:25.056302 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:25.056029 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:30:25.056302 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:25.056075 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:31:29.056059304 +0000 UTC m=+162.750402261 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:30:25.056302 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:25.056017 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:30:25.056302 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:25.056165 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:31:29.056151527 +0000 UTC m=+162.750494484 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:30:29.105155 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:29.105118 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6kwjn" Jun 09 05:30:56.583639 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:30:56.583592 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:30:56.584128 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:56.583742 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 05:30:56.584128 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:30:56.583811 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs podName:879b1a42-fd63-4e8a-be9b-7886e17a8408 nodeName:}" failed. No retries permitted until 2026-06-09 05:32:58.583793786 +0000 UTC m=+252.278136745 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs") pod "network-metrics-daemon-wfj72" (UID: "879b1a42-fd63-4e8a-be9b-7886e17a8408") : secret "metrics-daemon-secret" not found Jun 09 05:31:24.193125 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:24.193078 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-864f978974-4b5sl" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" Jun 09 05:31:24.203454 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:24.203421 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8r8dx" podUID="40108820-3325-4e88-a8b1-056266249a9b" Jun 09 05:31:24.297591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:31:24.297560 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r8dx" Jun 09 05:31:24.297714 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:31:24.297560 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:31:25.815710 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:25.815674 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-wfj72" podUID="879b1a42-fd63-4e8a-be9b-7886e17a8408" Jun 09 05:31:29.118300 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:31:29.118262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:31:29.118782 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:31:29.118314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:31:29.118782 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:29.118431 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 05:31:29.118782 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:29.118435 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 05:31:29.118782 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:29.118453 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864f978974-4b5sl: secret "image-registry-tls" not found Jun 09 05:31:29.118782 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:29.118490 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls podName:40108820-3325-4e88-a8b1-056266249a9b nodeName:}" failed. No retries permitted until 2026-06-09 05:33:31.118476728 +0000 UTC m=+284.812819688 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls") pod "dns-default-8r8dx" (UID: "40108820-3325-4e88-a8b1-056266249a9b") : secret "dns-default-metrics-tls" not found Jun 09 05:31:29.118782 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:31:29.118504 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls podName:8eabb6ca-08b7-4322-9a7b-ca108ac80a44 nodeName:}" failed. No retries permitted until 2026-06-09 05:33:31.118496715 +0000 UTC m=+284.812839673 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls") pod "image-registry-864f978974-4b5sl" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44") : secret "image-registry-tls" not found Jun 09 05:31:40.796771 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:31:40.794058 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:32:03.499143 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:03.499107 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:11.133778 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:11.133738 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" podUID="f5669941-51c5-4cc6-a407-42660f34eda1" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jun 09 05:32:11.408848 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:11.408768 2567 generic.go:356] "Generic (PLEG): container finished" podID="f5669941-51c5-4cc6-a407-42660f34eda1" containerID="cbe8f75515da7f9da47f99a90ef70023c845f0c895f19084ce7aed7f9fe0d076" exitCode=1 Jun 09 05:32:11.408848 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:11.408828 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" event={"ID":"f5669941-51c5-4cc6-a407-42660f34eda1","Type":"ContainerDied","Data":"cbe8f75515da7f9da47f99a90ef70023c845f0c895f19084ce7aed7f9fe0d076"} Jun 09 05:32:11.409120 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:11.409108 2567 scope.go:117] "RemoveContainer" containerID="cbe8f75515da7f9da47f99a90ef70023c845f0c895f19084ce7aed7f9fe0d076" Jun 09 05:32:12.412729 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:12.412696 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" event={"ID":"f5669941-51c5-4cc6-a407-42660f34eda1","Type":"ContainerStarted","Data":"d2dbb4a26a791fd29db48249eafa1db86f69fb2b7ac8f4d5735f8a53f9861178"} Jun 09 05:32:12.413173 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:12.413072 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:32:12.413609 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:12.413591 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9bdd66bf8-22jtk" Jun 09 05:32:27.629976 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.629938 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-t4t2t"] Jun 09 05:32:27.631725 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.631708 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.633967 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.633941 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 09 05:32:27.634275 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.634261 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fkbng\"" Jun 09 05:32:27.635196 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.635176 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 09 05:32:27.635196 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.635195 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 09 05:32:27.635435 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.635183 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 09 05:32:27.639953 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.639934 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 09 05:32:27.644495 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.644478 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-t4t2t"] Jun 09 05:32:27.718262 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.718230 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8850590e-542e-45ca-a2a7-8ed744959331-serving-cert\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.718262 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.718266 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8850590e-542e-45ca-a2a7-8ed744959331-trusted-ca\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.718523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.718288 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8850590e-542e-45ca-a2a7-8ed744959331-config\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.718523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.718405 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xpmb\" (UniqueName: \"kubernetes.io/projected/8850590e-542e-45ca-a2a7-8ed744959331-kube-api-access-8xpmb\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.733185 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.733153 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c4bdb859-fr94v"] Jun 09 05:32:27.735018 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.735003 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.737257 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737238 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b4g7d\"" Jun 09 05:32:27.737556 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737524 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 09 05:32:27.737556 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737542 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 09 05:32:27.737709 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737542 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 09 05:32:27.737709 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737544 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 05:32:27.737709 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737685 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 09 05:32:27.737976 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.737960 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 05:32:27.748757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.748737 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c4bdb859-fr94v"] Jun 09 05:32:27.819832 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8850590e-542e-45ca-a2a7-8ed744959331-serving-cert\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.819832 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819834 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8850590e-542e-45ca-a2a7-8ed744959331-trusted-ca\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.820068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8850590e-542e-45ca-a2a7-8ed744959331-config\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.820068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819889 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2qww\" (UniqueName: \"kubernetes.io/projected/433e22e2-9c9b-4542-988b-9572b5ad189f-kube-api-access-j2qww\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.820068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819927 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.820068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-default-certificate\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.820068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.819992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.820281 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.820073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xpmb\" (UniqueName: \"kubernetes.io/projected/8850590e-542e-45ca-a2a7-8ed744959331-kube-api-access-8xpmb\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.820281 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.820120 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-stats-auth\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.820537 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.820508 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8850590e-542e-45ca-a2a7-8ed744959331-config\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.820915 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.820891 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8850590e-542e-45ca-a2a7-8ed744959331-trusted-ca\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.822205 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.822185 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8850590e-542e-45ca-a2a7-8ed744959331-serving-cert\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.832903 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.832879 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xpmb\" (UniqueName: \"kubernetes.io/projected/8850590e-542e-45ca-a2a7-8ed744959331-kube-api-access-8xpmb\") pod \"console-operator-668fdbf85c-t4t2t\" (UID: \"8850590e-542e-45ca-a2a7-8ed744959331\") " pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.868304 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.868275 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:27.921316 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.921254 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2qww\" (UniqueName: \"kubernetes.io/projected/433e22e2-9c9b-4542-988b-9572b5ad189f-kube-api-access-j2qww\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.921316 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.921289 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.921316 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.921317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-default-certificate\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.921557 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.921333 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.921557 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:27.921434 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 05:32:27.921557 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.921458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-stats-auth\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.921557 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:27.921475 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:28.42145538 +0000 UTC m=+222.115798354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : configmap references non-existent config key: service-ca.crt Jun 09 05:32:27.921557 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:27.921509 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:28.421498193 +0000 UTC m=+222.115841153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : secret "router-metrics-certs-default" not found Jun 09 05:32:27.923672 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.923643 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-stats-auth\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.923672 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.923669 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-default-certificate\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.929539 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.929514 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2qww\" (UniqueName: \"kubernetes.io/projected/433e22e2-9c9b-4542-988b-9572b5ad189f-kube-api-access-j2qww\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:27.941845 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.941829 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:27.968096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:27.968052 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:28.057646 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:28.057554 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-t4t2t"] Jun 09 05:32:28.060024 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:32:28.060001 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8850590e_542e_45ca_a2a7_8ed744959331.slice/crio-342195c4268d980e5cec00e7908df9b87d9c41472fc18dbd121be9adde3deb50 WatchSource:0}: Error finding container 342195c4268d980e5cec00e7908df9b87d9c41472fc18dbd121be9adde3deb50: Status 404 returned error can't find the container with id 342195c4268d980e5cec00e7908df9b87d9c41472fc18dbd121be9adde3deb50 Jun 09 05:32:28.425096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:28.425058 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:28.425096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:28.425103 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:28.425323 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:28.425215 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 05:32:28.425323 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:28.425234 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:29.425217964 +0000 UTC m=+223.119560922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : configmap references non-existent config key: service-ca.crt Jun 09 05:32:28.425323 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:28.425275 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:29.425261551 +0000 UTC m=+223.119604507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : secret "router-metrics-certs-default" not found Jun 09 05:32:28.450971 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:28.450937 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" event={"ID":"8850590e-542e-45ca-a2a7-8ed744959331","Type":"ContainerStarted","Data":"342195c4268d980e5cec00e7908df9b87d9c41472fc18dbd121be9adde3deb50"} Jun 09 05:32:29.434664 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:29.434628 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:29.435142 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:29.434687 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:29.435142 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:29.434804 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 05:32:29.435142 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:29.434879 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:31.434863429 +0000 UTC m=+225.129206390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : secret "router-metrics-certs-default" not found Jun 09 05:32:29.435142 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:29.434896 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:31.434889587 +0000 UTC m=+225.129232544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : configmap references non-existent config key: service-ca.crt Jun 09 05:32:30.036699 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:30.036640 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:30.137830 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:30.137791 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:31.454182 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:31.454127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:31.454668 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:31.454257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:31.454668 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:31.454291 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:35.454272974 +0000 UTC m=+229.148615930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : configmap references non-existent config key: service-ca.crt Jun 09 05:32:31.454668 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:31.454377 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 05:32:31.454668 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:31.454462 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:35.454444608 +0000 UTC m=+229.148787568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : secret "router-metrics-certs-default" not found Jun 09 05:32:31.460211 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:31.460191 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/0.log" Jun 09 05:32:31.460291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:31.460228 2567 generic.go:356] "Generic (PLEG): container finished" podID="8850590e-542e-45ca-a2a7-8ed744959331" containerID="159b37ec824a68b388db72863a6a0c4b6b01887f325f612ee63ef42dd46e6b04" exitCode=255 Jun 09 05:32:31.460291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:31.460271 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" event={"ID":"8850590e-542e-45ca-a2a7-8ed744959331","Type":"ContainerDied","Data":"159b37ec824a68b388db72863a6a0c4b6b01887f325f612ee63ef42dd46e6b04"} Jun 09 05:32:31.460511 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:31.460497 2567 scope.go:117] "RemoveContainer" containerID="159b37ec824a68b388db72863a6a0c4b6b01887f325f612ee63ef42dd46e6b04" Jun 09 05:32:32.464046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.464016 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:32:32.464424 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.464375 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/0.log" Jun 09 05:32:32.464483 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.464422 2567 generic.go:356] "Generic (PLEG): container finished" podID="8850590e-542e-45ca-a2a7-8ed744959331" containerID="1fa779bedf059111c93f6a9b678837d2fca8f856c6873a9f5b5b6d686943cdf3" exitCode=255 Jun 09 05:32:32.464483 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.464471 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" event={"ID":"8850590e-542e-45ca-a2a7-8ed744959331","Type":"ContainerDied","Data":"1fa779bedf059111c93f6a9b678837d2fca8f856c6873a9f5b5b6d686943cdf3"} Jun 09 05:32:32.464549 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.464514 2567 scope.go:117] "RemoveContainer" containerID="159b37ec824a68b388db72863a6a0c4b6b01887f325f612ee63ef42dd46e6b04" Jun 09 05:32:32.464739 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.464699 2567 scope.go:117] "RemoveContainer" containerID="1fa779bedf059111c93f6a9b678837d2fca8f856c6873a9f5b5b6d686943cdf3" Jun 09 05:32:32.464892 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:32.464872 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-t4t2t_openshift-console-operator(8850590e-542e-45ca-a2a7-8ed744959331)\"" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" podUID="8850590e-542e-45ca-a2a7-8ed744959331" Jun 09 05:32:32.998376 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:32.998344 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.053378 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.053344 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.131230 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.131203 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.181003 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.180974 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.469825 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.469798 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:32:33.470212 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.470104 2567 scope.go:117] "RemoveContainer" containerID="1fa779bedf059111c93f6a9b678837d2fca8f856c6873a9f5b5b6d686943cdf3" Jun 09 05:32:33.470251 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:33.470239 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-t4t2t_openshift-console-operator(8850590e-542e-45ca-a2a7-8ed744959331)\"" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" podUID="8850590e-542e-45ca-a2a7-8ed744959331" Jun 09 05:32:33.559160 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.559122 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.725379 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.725305 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.828591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.828559 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:33.926579 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:33.926551 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:34.721701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.721671 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx"] Jun 09 05:32:34.724033 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.724017 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" Jun 09 05:32:34.726152 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.725714 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l6qxp_1b6e3541-7a83-41ea-bdea-f8e2f4787005/dns-node-resolver/0.log" Jun 09 05:32:34.727036 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.726703 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-x7fc8\"" Jun 09 05:32:34.727036 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.726764 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 09 05:32:34.727036 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.726707 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 09 05:32:34.731871 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.731848 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx"] Jun 09 05:32:34.759181 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.759138 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:34.811904 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.811861 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:34.878543 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.878510 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5924v\" (UniqueName: \"kubernetes.io/projected/a71c6eb8-6b46-4747-bf51-9e38450fa9df-kube-api-access-5924v\") pod \"migrator-955ffdf5f-bs8kx\" (UID: \"a71c6eb8-6b46-4747-bf51-9e38450fa9df\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" Jun 09 05:32:34.979233 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.979158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5924v\" (UniqueName: \"kubernetes.io/projected/a71c6eb8-6b46-4747-bf51-9e38450fa9df-kube-api-access-5924v\") pod \"migrator-955ffdf5f-bs8kx\" (UID: \"a71c6eb8-6b46-4747-bf51-9e38450fa9df\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" Jun 09 05:32:34.986923 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:34.986903 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5924v\" (UniqueName: \"kubernetes.io/projected/a71c6eb8-6b46-4747-bf51-9e38450fa9df-kube-api-access-5924v\") pod \"migrator-955ffdf5f-bs8kx\" (UID: \"a71c6eb8-6b46-4747-bf51-9e38450fa9df\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" Jun 09 05:32:35.034978 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.034950 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" Jun 09 05:32:35.149957 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.149921 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx"] Jun 09 05:32:35.153373 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:32:35.153351 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71c6eb8_6b46_4747_bf51_9e38450fa9df.slice/crio-b94ff9c58fa24289017950ca1a604f1df79c4a6ee829e913e4648e8c14fe18a7 WatchSource:0}: Error finding container b94ff9c58fa24289017950ca1a604f1df79c4a6ee829e913e4648e8c14fe18a7: Status 404 returned error can't find the container with id b94ff9c58fa24289017950ca1a604f1df79c4a6ee829e913e4648e8c14fe18a7 Jun 09 05:32:35.429745 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.429712 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-6qjjq"] Jun 09 05:32:35.431595 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.431580 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.433813 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.433783 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 09 05:32:35.433916 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.433882 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 09 05:32:35.434911 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.434892 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8fjgh\"" Jun 09 05:32:35.435016 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.434897 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 09 05:32:35.435016 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.434899 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 09 05:32:35.441480 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.441459 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-6qjjq"] Jun 09 05:32:35.468215 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.468186 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:35.476253 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.476230 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" event={"ID":"a71c6eb8-6b46-4747-bf51-9e38450fa9df","Type":"ContainerStarted","Data":"b94ff9c58fa24289017950ca1a604f1df79c4a6ee829e913e4648e8c14fe18a7"} Jun 09 05:32:35.483602 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.483578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:35.483687 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.483610 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:35.483738 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:35.483721 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 05:32:35.483772 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:35.483747 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:43.483730106 +0000 UTC m=+237.178073069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : configmap references non-existent config key: service-ca.crt Jun 09 05:32:35.483772 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:35.483766 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:43.483755119 +0000 UTC m=+237.178098093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : secret "router-metrics-certs-default" not found Jun 09 05:32:35.517887 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.517864 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:35.584237 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.584215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-signing-key\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.584316 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.584253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-signing-cabundle\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.584350 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.584323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chpwt\" (UniqueName: \"kubernetes.io/projected/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-kube-api-access-chpwt\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.685264 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.685198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-signing-cabundle\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.685381 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.685305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chpwt\" (UniqueName: \"kubernetes.io/projected/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-kube-api-access-chpwt\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.685381 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.685350 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-signing-key\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.685967 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.685944 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-signing-cabundle\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.687730 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.687708 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-signing-key\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.693184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.693160 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chpwt\" (UniqueName: \"kubernetes.io/projected/5c722e92-a2d1-4ba2-8fbe-ae4d3c136083-kube-api-access-chpwt\") pod \"service-ca-7d74d578b8-6qjjq\" (UID: \"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083\") " pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.740339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.740313 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" Jun 09 05:32:35.876730 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.876702 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-6qjjq"] Jun 09 05:32:35.879197 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:32:35.879174 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c722e92_a2d1_4ba2_8fbe_ae4d3c136083.slice/crio-6e66c5fa37211b58b7aaec42ba22555a3bab0223437179e9b7e9030d441077bb WatchSource:0}: Error finding container 6e66c5fa37211b58b7aaec42ba22555a3bab0223437179e9b7e9030d441077bb: Status 404 returned error can't find the container with id 6e66c5fa37211b58b7aaec42ba22555a3bab0223437179e9b7e9030d441077bb Jun 09 05:32:35.923131 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:35.923111 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vqxk4_a5af69f3-c1ad-4531-b392-4313706bcb16/node-ca/0.log" Jun 09 05:32:36.279505 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:36.279468 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:36.481287 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:36.481244 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" event={"ID":"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083","Type":"ContainerStarted","Data":"6e66c5fa37211b58b7aaec42ba22555a3bab0223437179e9b7e9030d441077bb"} Jun 09 05:32:36.725984 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:36.725932 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dhhmg_c9a9a461-a7dc-4aef-9008-e2418f03da29/serve-healthcheck-canary/0.log" Jun 09 05:32:37.487116 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:37.487082 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" event={"ID":"a71c6eb8-6b46-4747-bf51-9e38450fa9df","Type":"ContainerStarted","Data":"54ee36ae8bc19013fc1a187d30bf4a071f0d3675a9013e386d64dde287cc7cb4"} Jun 09 05:32:37.487582 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:37.487137 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" event={"ID":"a71c6eb8-6b46-4747-bf51-9e38450fa9df","Type":"ContainerStarted","Data":"87e76dd7ea4cdbf4014c7c8630fbd095e65fee68579c4c3d1bf407c406778cc3"} Jun 09 05:32:37.505276 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:37.505218 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-bs8kx" podStartSLOduration=2.130432208 podStartE2EDuration="3.505200021s" totalImagesPullingTime="1.374767813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:32:37.504233627 +0000 UTC m=+231.198576618" watchObservedRunningTime="2026-06-09 05:32:37.505200021 +0000 UTC m=+231.199543000" Jun 09 05:32:37.942471 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:37.942431 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:37.942628 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:37.942492 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:37.942933 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:37.942914 2567 scope.go:117] "RemoveContainer" containerID="1fa779bedf059111c93f6a9b678837d2fca8f856c6873a9f5b5b6d686943cdf3" Jun 09 05:32:37.943115 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:37.943096 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-t4t2t_openshift-console-operator(8850590e-542e-45ca-a2a7-8ed744959331)\"" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" podUID="8850590e-542e-45ca-a2a7-8ed744959331" Jun 09 05:32:38.494404 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:38.491838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" event={"ID":"5c722e92-a2d1-4ba2-8fbe-ae4d3c136083","Type":"ContainerStarted","Data":"18bd3e1f6b03bd1061595ee73abb8928f0798c8eda2781c860cdf547ef85e9f9"} Jun 09 05:32:38.513834 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:38.513788 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-6qjjq" podStartSLOduration=1.75141469 podStartE2EDuration="3.513774377s" totalImagesPullingTime="1.762359687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:32:38.5120787 +0000 UTC m=+232.206421680" watchObservedRunningTime="2026-06-09 05:32:38.513774377 +0000 UTC m=+232.208117355" Jun 09 05:32:38.646480 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:38.646444 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:38.993294 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:38.993257 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:39.147306 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:39.147271 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:39.279021 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:39.278912 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:39.331460 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:39.331419 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:39.379509 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:39.379467 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:39.512603 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:39.512565 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:39.691976 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:39.691942 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:40.690154 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:40.690121 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:40.890305 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:40.890268 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:41.286165 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:41.286136 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:41.492137 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:41.492109 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:42.086586 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:42.086555 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:42.592861 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:42.592828 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:42.693916 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:42.693878 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:42.886273 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:42.886238 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:42.936496 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:42.936467 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:43.295569 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:43.295481 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:43.410775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:43.410746 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:43.486984 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:43.486952 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:43.549339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:43.549257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:43.549339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:43.549299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:43.549568 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:43.549358 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 09 05:32:43.549568 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:43.549432 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:59.549417911 +0000 UTC m=+253.243760868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : secret "router-metrics-certs-default" not found Jun 09 05:32:43.549568 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:32:43.549463 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle podName:433e22e2-9c9b-4542-988b-9572b5ad189f nodeName:}" failed. No retries permitted until 2026-06-09 05:32:59.549448658 +0000 UTC m=+253.243791616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle") pod "router-default-7c4bdb859-fr94v" (UID: "433e22e2-9c9b-4542-988b-9572b5ad189f") : configmap references non-existent config key: service-ca.crt Jun 09 05:32:44.087534 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:44.087501 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:44.491134 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:44.491100 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:44.692437 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:44.692408 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:45.089638 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:45.089600 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:45.519675 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:45.519643 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:45.921401 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:45.921357 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:46.020945 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:46.020919 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:46.311033 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:46.310950 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:46.489172 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:46.489140 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:46.693868 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:46.693836 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:46.890544 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:46.890508 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:47.887166 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:47.887138 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:48.289531 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:48.289447 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:48.489976 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:48.489935 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:48.687199 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:48.687172 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:48.886299 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:48.886267 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:49.491591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:49.491552 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:49.694211 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:49.694184 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:50.793683 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:50.793654 2567 scope.go:117] "RemoveContainer" containerID="1fa779bedf059111c93f6a9b678837d2fca8f856c6873a9f5b5b6d686943cdf3" Jun 09 05:32:51.351503 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.351466 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:51.398401 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.398355 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:51.451353 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.451327 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:51.535525 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.535495 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:32:51.535701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.535543 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" event={"ID":"8850590e-542e-45ca-a2a7-8ed744959331","Type":"ContainerStarted","Data":"768a362ecde58efba14a7e6ed29204d153af6f60606b9d31106e83b0354cc176"} Jun 09 05:32:51.535846 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.535826 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:51.540442 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.540418 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" Jun 09 05:32:51.551162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.551121 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-t4t2t" podStartSLOduration=22.035194622 podStartE2EDuration="24.551109067s" totalImagesPullingTime="2.515914445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:32:51.550214502 +0000 UTC m=+245.244557505" watchObservedRunningTime="2026-06-09 05:32:51.551109067 +0000 UTC m=+245.245452046" Jun 09 05:32:51.649999 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.649970 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:51.845174 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:51.845145 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:52.031999 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:52.031923 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:52.133450 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:52.133423 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:52.457520 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:52.457487 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:52.558346 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:52.558318 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:52.658487 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:52.658450 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:52.705307 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:52.705279 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:53.041639 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:53.041611 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:53.242967 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:53.242932 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:53.572269 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:53.572231 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:54.269508 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.269472 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b5g88"] Jun 09 05:32:54.272701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.272683 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.275067 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.275041 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 05:32:54.275205 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.275158 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 05:32:54.275205 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.275167 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w5x62\"" Jun 09 05:32:54.275205 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.275168 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 05:32:54.276089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.276066 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 05:32:54.283688 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.283657 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b5g88"] Jun 09 05:32:54.314454 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.314430 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-kpcfx"] Jun 09 05:32:54.317580 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.317562 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:32:54.319979 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.319957 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rk4nl\"" Jun 09 05:32:54.320089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.319957 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 09 05:32:54.320149 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.320112 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 09 05:32:54.327109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.327089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-kpcfx"] Jun 09 05:32:54.335153 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.335122 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e92aa975-0d99-4d99-bee0-a1e94cef975f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.335265 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.335188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e92aa975-0d99-4d99-bee0-a1e94cef975f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.335265 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.335234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e92aa975-0d99-4d99-bee0-a1e94cef975f-data-volume\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.335359 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.335299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgjfq\" (UniqueName: \"kubernetes.io/projected/cfc3eea4-b827-41a8-b62d-3ce6fc89ef64-kube-api-access-qgjfq\") pod \"downloads-6657875f97-kpcfx\" (UID: \"cfc3eea4-b827-41a8-b62d-3ce6fc89ef64\") " pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:32:54.335359 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.335339 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e92aa975-0d99-4d99-bee0-a1e94cef975f-crio-socket\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.335471 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.335430 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjmng\" (UniqueName: \"kubernetes.io/projected/e92aa975-0d99-4d99-bee0-a1e94cef975f-kube-api-access-zjmng\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.375586 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.375552 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:54.435973 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.435941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjmng\" (UniqueName: \"kubernetes.io/projected/e92aa975-0d99-4d99-bee0-a1e94cef975f-kube-api-access-zjmng\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436134 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.435988 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e92aa975-0d99-4d99-bee0-a1e94cef975f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436134 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e92aa975-0d99-4d99-bee0-a1e94cef975f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436251 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e92aa975-0d99-4d99-bee0-a1e94cef975f-data-volume\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436251 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgjfq\" (UniqueName: \"kubernetes.io/projected/cfc3eea4-b827-41a8-b62d-3ce6fc89ef64-kube-api-access-qgjfq\") pod \"downloads-6657875f97-kpcfx\" (UID: \"cfc3eea4-b827-41a8-b62d-3ce6fc89ef64\") " pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:32:54.436251 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e92aa975-0d99-4d99-bee0-a1e94cef975f-crio-socket\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436365 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436299 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e92aa975-0d99-4d99-bee0-a1e94cef975f-crio-socket\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436581 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436556 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e92aa975-0d99-4d99-bee0-a1e94cef975f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.436581 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.436556 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e92aa975-0d99-4d99-bee0-a1e94cef975f-data-volume\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.438452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.438427 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e92aa975-0d99-4d99-bee0-a1e94cef975f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.445563 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.445537 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgjfq\" (UniqueName: \"kubernetes.io/projected/cfc3eea4-b827-41a8-b62d-3ce6fc89ef64-kube-api-access-qgjfq\") pod \"downloads-6657875f97-kpcfx\" (UID: \"cfc3eea4-b827-41a8-b62d-3ce6fc89ef64\") " pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:32:54.445693 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.445676 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjmng\" (UniqueName: \"kubernetes.io/projected/e92aa975-0d99-4d99-bee0-a1e94cef975f-kube-api-access-zjmng\") pod \"insights-runtime-extractor-b5g88\" (UID: \"e92aa975-0d99-4d99-bee0-a1e94cef975f\") " pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.475088 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.475062 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:54.581296 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.581221 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b5g88" Jun 09 05:32:54.627624 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.627591 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:32:54.649010 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.648807 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:54.700868 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.700792 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:54.734119 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.734052 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b5g88"] Jun 09 05:32:54.736120 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:32:54.736096 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92aa975_0d99_4d99_bee0_a1e94cef975f.slice/crio-3f9f00948201e0654652b7dc95989ff668e837d70c7639c4356a67a7fc39d6ed WatchSource:0}: Error finding container 3f9f00948201e0654652b7dc95989ff668e837d70c7639c4356a67a7fc39d6ed: Status 404 returned error can't find the container with id 3f9f00948201e0654652b7dc95989ff668e837d70c7639c4356a67a7fc39d6ed Jun 09 05:32:54.771584 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:54.771561 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-kpcfx"] Jun 09 05:32:54.774043 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:32:54.774015 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc3eea4_b827_41a8_b62d_3ce6fc89ef64.slice/crio-04ca70fdd74cd84587fc900fbfda036d52f145993494c155eb3e489fff4d3270 WatchSource:0}: Error finding container 04ca70fdd74cd84587fc900fbfda036d52f145993494c155eb3e489fff4d3270: Status 404 returned error can't find the container with id 04ca70fdd74cd84587fc900fbfda036d52f145993494c155eb3e489fff4d3270 Jun 09 05:32:55.040818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:55.040785 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:55.093336 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:55.093296 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:32:55.553547 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:55.553506 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b5g88" event={"ID":"e92aa975-0d99-4d99-bee0-a1e94cef975f","Type":"ContainerStarted","Data":"a77669591b1852fbed94ab4ce7db80cc9fbb56a2e44c05e349c1b87450e0aee8"} Jun 09 05:32:55.553994 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:55.553577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b5g88" event={"ID":"e92aa975-0d99-4d99-bee0-a1e94cef975f","Type":"ContainerStarted","Data":"90aa31a12e8053f35bb3431918398a51dd8dd0bedcf16647565f67d5b35bafd5"} Jun 09 05:32:55.553994 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:55.553610 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b5g88" event={"ID":"e92aa975-0d99-4d99-bee0-a1e94cef975f","Type":"ContainerStarted","Data":"3f9f00948201e0654652b7dc95989ff668e837d70c7639c4356a67a7fc39d6ed"} Jun 09 05:32:55.554731 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:55.554701 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-kpcfx" event={"ID":"cfc3eea4-b827-41a8-b62d-3ce6fc89ef64","Type":"ContainerStarted","Data":"04ca70fdd74cd84587fc900fbfda036d52f145993494c155eb3e489fff4d3270"} Jun 09 05:32:57.564789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:57.564749 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b5g88" event={"ID":"e92aa975-0d99-4d99-bee0-a1e94cef975f","Type":"ContainerStarted","Data":"ab092826cf1b1df55b12870b14460fcd826b9b2905bb8756f8031dbb561e6949"} Jun 09 05:32:57.581091 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:57.581027 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b5g88" podStartSLOduration=1.4465861850000001 podStartE2EDuration="3.581010661s" totalImagesPullingTime="2.134424476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:32:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:32:57.579798372 +0000 UTC m=+251.274141350" watchObservedRunningTime="2026-06-09 05:32:57.581010661 +0000 UTC m=+251.275353640" Jun 09 05:32:58.669303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:58.669261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:32:58.672029 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:58.672002 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/879b1a42-fd63-4e8a-be9b-7886e17a8408-metrics-certs\") pod \"network-metrics-daemon-wfj72\" (UID: \"879b1a42-fd63-4e8a-be9b-7886e17a8408\") " pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:32:58.797349 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:58.797318 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tg4g2\"" Jun 09 05:32:58.805368 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:58.805341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wfj72" Jun 09 05:32:58.949499 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:58.949424 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wfj72"] Jun 09 05:32:58.952297 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:32:58.952266 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879b1a42_fd63_4e8a_be9b_7886e17a8408.slice/crio-4316cbc373bc958c7a5f6f02e4a32a0993843ed9e8805deb1bc726d7a8901a83 WatchSource:0}: Error finding container 4316cbc373bc958c7a5f6f02e4a32a0993843ed9e8805deb1bc726d7a8901a83: Status 404 returned error can't find the container with id 4316cbc373bc958c7a5f6f02e4a32a0993843ed9e8805deb1bc726d7a8901a83 Jun 09 05:32:59.574011 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.573971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wfj72" event={"ID":"879b1a42-fd63-4e8a-be9b-7886e17a8408","Type":"ContainerStarted","Data":"4316cbc373bc958c7a5f6f02e4a32a0993843ed9e8805deb1bc726d7a8901a83"} Jun 09 05:32:59.577367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.577330 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:59.577504 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.577380 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:59.578117 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.578091 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433e22e2-9c9b-4542-988b-9572b5ad189f-service-ca-bundle\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:59.580264 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.580238 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433e22e2-9c9b-4542-988b-9572b5ad189f-metrics-certs\") pod \"router-default-7c4bdb859-fr94v\" (UID: \"433e22e2-9c9b-4542-988b-9572b5ad189f\") " pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:32:59.846717 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.846637 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b4g7d\"" Jun 09 05:32:59.854650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:32:59.854623 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:33:00.072100 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.072064 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:00.172087 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.172060 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:00.188896 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.188865 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c4bdb859-fr94v"] Jun 09 05:33:00.191344 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:00.191319 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433e22e2_9c9b_4542_988b_9572b5ad189f.slice/crio-fac8100715a6853f37eea866c7c7f45eb124e7721a3a6efdf76ba793998272e3 WatchSource:0}: Error finding container fac8100715a6853f37eea866c7c7f45eb124e7721a3a6efdf76ba793998272e3: Status 404 returned error can't find the container with id fac8100715a6853f37eea866c7c7f45eb124e7721a3a6efdf76ba793998272e3 Jun 09 05:33:00.579587 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.579545 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wfj72" event={"ID":"879b1a42-fd63-4e8a-be9b-7886e17a8408","Type":"ContainerStarted","Data":"0c976df9bf5ece35b169acb1bc9a6657d12995e36412ccd53e603670575a1cbf"} Jun 09 05:33:00.579768 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.579609 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wfj72" event={"ID":"879b1a42-fd63-4e8a-be9b-7886e17a8408","Type":"ContainerStarted","Data":"c496b680900df8b377926c20eeebd6bdcf5f2d96c62b80cd90c3930940de0dc4"} Jun 09 05:33:00.581267 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.581238 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c4bdb859-fr94v" event={"ID":"433e22e2-9c9b-4542-988b-9572b5ad189f","Type":"ContainerStarted","Data":"833c7e92674a3ce3214c8bbdee1d9f1742eb952d229875e942550baaf2fcaf68"} Jun 09 05:33:00.581405 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.581284 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c4bdb859-fr94v" event={"ID":"433e22e2-9c9b-4542-988b-9572b5ad189f","Type":"ContainerStarted","Data":"fac8100715a6853f37eea866c7c7f45eb124e7721a3a6efdf76ba793998272e3"} Jun 09 05:33:00.594314 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.594269 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wfj72" podStartSLOduration=253.460757884 podStartE2EDuration="4m14.594255775s" totalImagesPullingTime="1.133497891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:00.593161618 +0000 UTC m=+254.287504601" watchObservedRunningTime="2026-06-09 05:33:00.594255775 +0000 UTC m=+254.288598757" Jun 09 05:33:00.609613 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.609572 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c4bdb859-fr94v" podStartSLOduration=33.609554505 podStartE2EDuration="33.609554505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:00.609509549 +0000 UTC m=+254.303852529" watchObservedRunningTime="2026-06-09 05:33:00.609554505 +0000 UTC m=+254.303897485" Jun 09 05:33:00.674311 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.674229 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:00.724142 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.724115 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:00.855762 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.855671 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:33:00.858550 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:00.858528 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:33:01.584751 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:01.584718 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:33:01.586007 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:01.585984 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c4bdb859-fr94v" Jun 09 05:33:01.774214 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:01.774184 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:01.876666 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:01.876633 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:01.962190 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:01.962158 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:02.172056 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:02.171906 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:02.225971 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:02.225935 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:02.500122 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:02.500046 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:03.066822 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:03.066782 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:04.869717 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:04.869688 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:04.934733 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:04.934696 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:04.993263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:04.993236 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:05.211434 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:05.211343 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:07.530239 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:07.530151 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:08.663748 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:08.663711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:08.713216 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:08.713166 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.079800 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.079714 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.175637 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.175601 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.290285 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.290217 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.350911 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.350773 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vvfb2"] Jun 09 05:33:09.354474 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.354444 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.358604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.358559 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 05:33:09.359227 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.359037 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 05:33:09.359709 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.359684 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wgd4p\"" Jun 09 05:33:09.360203 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.360171 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 05:33:09.360283 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.360255 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 05:33:09.360700 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.360680 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 05:33:09.360919 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.360903 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 05:33:09.375981 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.375958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.436357 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.436323 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.460869 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.460837 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6rpg\" (UniqueName: \"kubernetes.io/projected/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-kube-api-access-v6rpg\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461016 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.460884 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-root\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461016 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.460939 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-textfile\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461140 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.461018 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461140 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.461055 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-metrics-client-ca\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461140 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.461082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461140 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.461110 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-tls\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.461176 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-sys\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.461303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.461209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-wtmp\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.562973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-root\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563045 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-textfile\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563080 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563108 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-metrics-client-ca\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563131 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-tls\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563190 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-sys\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-wtmp\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6rpg\" (UniqueName: \"kubernetes.io/projected/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-kube-api-access-v6rpg\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563923 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563656 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-root\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.563981 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.563963 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-textfile\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.564501 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.564480 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.564961 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.564942 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-metrics-client-ca\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.565339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.565315 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-sys\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.565993 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.565969 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-wtmp\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.568695 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.568644 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-tls\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.569591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.569551 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.571441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.571420 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6rpg\" (UniqueName: \"kubernetes.io/projected/fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c-kube-api-access-v6rpg\") pod \"node-exporter-vvfb2\" (UID: \"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c\") " pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.591500 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.591471 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.667585 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.667551 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vvfb2" Jun 09 05:33:09.689830 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.689763 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.789855 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.789811 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.890667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.890619 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:09.950019 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:09.949932 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:10.040580 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:10.040539 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:10.143839 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:10.143805 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:10.304815 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:10.304733 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:10.401408 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:10.388533 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:10.638065 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:10.638029 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:12.182298 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:12.182266 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc5ebf5_bcbc_4e18_9110_d1c2b33e965c.slice/crio-c88f4edb6788b5a7c7fd4080118cf5f663cc2ec91defb47c72dbfdd405b8ab75 WatchSource:0}: Error finding container c88f4edb6788b5a7c7fd4080118cf5f663cc2ec91defb47c72dbfdd405b8ab75: Status 404 returned error can't find the container with id c88f4edb6788b5a7c7fd4080118cf5f663cc2ec91defb47c72dbfdd405b8ab75 Jun 09 05:33:12.241544 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.241517 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:12.306342 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.306313 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:12.313223 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.313200 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b676678c7-l7dkh"] Jun 09 05:33:12.317573 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.317556 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.320129 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320095 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 09 05:33:12.320217 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320098 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 09 05:33:12.320291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320212 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 09 05:33:12.320291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320220 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j6snm\"" Jun 09 05:33:12.320291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320230 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-38n3n5v8vaeig\"" Jun 09 05:33:12.320438 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320309 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 09 05:33:12.320521 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.320507 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 09 05:33:12.328244 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.328224 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b676678c7-l7dkh"] Jun 09 05:33:12.384458 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.384428 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:12.386430 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386406 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fgpn\" (UniqueName: \"kubernetes.io/projected/6dff0626-9f48-4d33-b8da-77f7c9b29e27-kube-api-access-7fgpn\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386574 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386574 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386496 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386685 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386597 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dff0626-9f48-4d33-b8da-77f7c9b29e27-metrics-client-ca\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386685 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386629 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386685 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386677 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386796 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386712 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-grpc-tls\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.386796 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.386753 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-tls\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.488617 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-tls\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.488790 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgpn\" (UniqueName: \"kubernetes.io/projected/6dff0626-9f48-4d33-b8da-77f7c9b29e27-kube-api-access-7fgpn\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.488790 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488656 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.488790 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488695 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.488790 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488731 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dff0626-9f48-4d33-b8da-77f7c9b29e27-metrics-client-ca\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.488790 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488762 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.489054 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488799 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.489054 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.488844 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-grpc-tls\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.491223 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.491188 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dff0626-9f48-4d33-b8da-77f7c9b29e27-metrics-client-ca\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.494662 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.494589 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.495857 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.495811 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-tls\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.496484 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.496436 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.497610 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.497533 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.498688 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.498231 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-grpc-tls\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.498688 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.498630 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dff0626-9f48-4d33-b8da-77f7c9b29e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.499797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.499759 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fgpn\" (UniqueName: \"kubernetes.io/projected/6dff0626-9f48-4d33-b8da-77f7c9b29e27-kube-api-access-7fgpn\") pod \"thanos-querier-5b676678c7-l7dkh\" (UID: \"6dff0626-9f48-4d33-b8da-77f7c9b29e27\") " pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.633080 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.633023 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-kpcfx" event={"ID":"cfc3eea4-b827-41a8-b62d-3ce6fc89ef64","Type":"ContainerStarted","Data":"fd5a7b93d12f191c8da7189247f6f131c07bd2476ae30d1b00a882221e206a84"} Jun 09 05:33:12.633507 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.633457 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:33:12.635360 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.635329 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvfb2" event={"ID":"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c","Type":"ContainerStarted","Data":"c88f4edb6788b5a7c7fd4080118cf5f663cc2ec91defb47c72dbfdd405b8ab75"} Jun 09 05:33:12.636122 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.636100 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:12.645510 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.645480 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-kpcfx" Jun 09 05:33:12.650111 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.649182 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-kpcfx" podStartSLOduration=1.156658444 podStartE2EDuration="18.649164271s" totalImagesPullingTime="17.492505827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:12.64813526 +0000 UTC m=+266.342478241" watchObservedRunningTime="2026-06-09 05:33:12.649164271 +0000 UTC m=+266.343507250" Jun 09 05:33:12.729013 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.728855 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:12.827138 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:12.827089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b676678c7-l7dkh"] Jun 09 05:33:13.064301 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:13.064227 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:13.069915 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:13.069887 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dff0626_9f48_4d33_b8da_77f7c9b29e27.slice/crio-65e3b5129909863a556eae0848784f1bda34c39a2303fcec5481cfdb01651961 WatchSource:0}: Error finding container 65e3b5129909863a556eae0848784f1bda34c39a2303fcec5481cfdb01651961: Status 404 returned error can't find the container with id 65e3b5129909863a556eae0848784f1bda34c39a2303fcec5481cfdb01651961 Jun 09 05:33:13.165065 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:13.165024 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:13.643454 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:13.643409 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"65e3b5129909863a556eae0848784f1bda34c39a2303fcec5481cfdb01651961"} Jun 09 05:33:13.647068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:13.646426 2567 generic.go:356] "Generic (PLEG): container finished" podID="fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c" containerID="c272f9fa8d1a7e8318c643e8a28fd526d344841cce83ce462b2ffb29437b4350" exitCode=0 Jun 09 05:33:13.647068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:13.647006 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvfb2" event={"ID":"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c","Type":"ContainerDied","Data":"c272f9fa8d1a7e8318c643e8a28fd526d344841cce83ce462b2ffb29437b4350"} Jun 09 05:33:14.114882 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.114789 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:14.121210 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.121177 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-x592k"] Jun 09 05:33:14.125952 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.125891 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.128853 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.128751 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 09 05:33:14.129247 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.129077 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hhd4m\"" Jun 09 05:33:14.134780 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.134682 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-x592k"] Jun 09 05:33:14.196411 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.196131 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:14.205596 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.204641 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.205596 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.204719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2bl\" (UniqueName: \"kubernetes.io/projected/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-kube-api-access-kt2bl\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.305422 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.305307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.305626 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.305424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2bl\" (UniqueName: \"kubernetes.io/projected/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-kube-api-access-kt2bl\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.305720 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:33:14.305640 2567 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 09 05:33:14.305783 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:33:14.305726 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-monitoring-plugin-cert podName:e0e540f5-fe13-48bc-9c18-84ed0ef1dae7 nodeName:}" failed. No retries permitted until 2026-06-09 05:33:14.805703705 +0000 UTC m=+268.500046664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-x592k" (UID: "e0e540f5-fe13-48bc-9c18-84ed0ef1dae7") : secret "monitoring-plugin-cert" not found Jun 09 05:33:14.317219 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.316501 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2bl\" (UniqueName: \"kubernetes.io/projected/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-kube-api-access-kt2bl\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.444122 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.444066 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:14.654648 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.654598 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvfb2" event={"ID":"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c","Type":"ContainerStarted","Data":"a3ded4452d56ced628371f47abfc4c73bf0800f67b3b21c4bfdc4cf7497ff793"} Jun 09 05:33:14.655116 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.654683 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvfb2" event={"ID":"fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c","Type":"ContainerStarted","Data":"8d757f0d20a0667f6fcb11d904d24ffdb5ce9e567a52f396ad44d344d04f0609"} Jun 09 05:33:14.671744 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.671686 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vvfb2" podStartSLOduration=4.660340585 podStartE2EDuration="5.671669274s" totalImagesPullingTime="932.585665ms" totalInitContainerRuntime="78.743024ms" isStatefulPod=true podCreationTimestamp="2026-06-09 05:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:14.670468397 +0000 UTC m=+268.364811377" watchObservedRunningTime="2026-06-09 05:33:14.671669274 +0000 UTC m=+268.366012254" Jun 09 05:33:14.810951 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.810869 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:14.813848 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:14.813794 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0e540f5-fe13-48bc-9c18-84ed0ef1dae7-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-x592k\" (UID: \"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:15.041588 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.041553 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:15.201262 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.201230 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-x592k"] Jun 09 05:33:15.352428 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:15.352327 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e540f5_fe13_48bc_9c18_84ed0ef1dae7.slice/crio-f97ad27d6a2307d8771ee9a915f753a8eb34aa2221a94a0bc28c10469262e7e4 WatchSource:0}: Error finding container f97ad27d6a2307d8771ee9a915f753a8eb34aa2221a94a0bc28c10469262e7e4: Status 404 returned error can't find the container with id f97ad27d6a2307d8771ee9a915f753a8eb34aa2221a94a0bc28c10469262e7e4 Jun 09 05:33:15.506082 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.506055 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:33:15.509687 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.509596 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:15.524056 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.524027 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:33:15.524220 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.524200 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.527023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.526997 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 05:33:15.527332 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.527312 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 05:33:15.527546 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.527525 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 05:33:15.529214 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.528292 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-301lng8agc3ov\"" Jun 09 05:33:15.529214 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.528513 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 05:33:15.529214 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.528883 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 05:33:15.529917 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.529545 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 05:33:15.529917 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.529563 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 05:33:15.530109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.529981 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-f7hb2\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.530341 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.530601 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.530824 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.530983 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.531014 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.531014 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 05:33:15.531297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.531108 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 05:33:15.533024 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.532750 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 05:33:15.619719 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619688 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.619830 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.619830 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.619830 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619793 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619848 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsnvf\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-kube-api-access-zsnvf\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619873 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.619963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620026 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620012 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620106 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-config\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620136 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-config-out\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620222 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620247 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-web-config\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620288 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620322 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620351 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.620491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.620379 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.659747 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.659713 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"46745a4ebe1ed69b35b39deccf906e87d9a1eb6273fd6dfc017273e1cf1dc955"} Jun 09 05:33:15.660102 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.659796 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"ae87058b1d24f7bf91d8fe4495bca852922f3cf368406d3a207872b125195104"} Jun 09 05:33:15.661067 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.661042 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" event={"ID":"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7","Type":"ContainerStarted","Data":"f97ad27d6a2307d8771ee9a915f753a8eb34aa2221a94a0bc28c10469262e7e4"} Jun 09 05:33:15.721501 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721419 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721501 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721471 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721639 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721507 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721639 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721639 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721606 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-config\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-config-out\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721729 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.721789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721784 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-web-config\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.721968 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.722004 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722330 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.722037 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722330 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.722066 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722330 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.722091 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.722330 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.722139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsnvf\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-kube-api-access-zsnvf\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.723537 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.723166 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.724972 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.724072 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.724972 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.724153 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.724972 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.724638 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.725901 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.725875 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.727921 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.727898 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.729479 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.729453 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.730078 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.730050 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.731476 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.730980 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-config\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.731476 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.731083 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.731476 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.731318 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.731476 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.731413 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.732115 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.731774 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.732277 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.732258 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-config-out\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.732696 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.732673 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.733082 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.733044 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.733327 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.733302 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.734044 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.734006 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-web-config\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.734890 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.734857 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsnvf\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-kube-api-access-zsnvf\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.735370 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.735351 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:15.866341 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:15.866265 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:16.050263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:16.050237 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:33:16.052399 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:16.052362 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea6cd74_bd49_44e8_abef_632333fcdb20.slice/crio-a15aa0c814f7326f759ca5263cee869e867f6fdb89a581bef440ce8c305f75ba WatchSource:0}: Error finding container a15aa0c814f7326f759ca5263cee869e867f6fdb89a581bef440ce8c305f75ba: Status 404 returned error can't find the container with id a15aa0c814f7326f759ca5263cee869e867f6fdb89a581bef440ce8c305f75ba Jun 09 05:33:16.670997 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:16.670865 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"8a9a38223b15b3aefb5b434766bef6153ecbdbf827448c1a79c1a319b0a15047"} Jun 09 05:33:16.672737 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:16.672660 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"a15aa0c814f7326f759ca5263cee869e867f6fdb89a581bef440ce8c305f75ba"} Jun 09 05:33:18.685674 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.685637 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"a5b0993074859033808916b6efbb1a534d8c9f6edbeda7690680f8c2101566e7"} Jun 09 05:33:18.686130 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.685716 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"b58679fa8b127f6a055eabc940738646c7a2883545914c0f501dd00bcbffd362"} Jun 09 05:33:18.686130 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.685762 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" event={"ID":"6dff0626-9f48-4d33-b8da-77f7c9b29e27","Type":"ContainerStarted","Data":"21c91775c0a533938fec346d1a0fb1b838d9f3b097dd1b7995a7a7e4e294b958"} Jun 09 05:33:18.686130 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.685811 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:18.687357 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.687331 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" event={"ID":"e0e540f5-fe13-48bc-9c18-84ed0ef1dae7","Type":"ContainerStarted","Data":"6e7078f5b05f5fb051621703840a5edb1190145e0af769a1ccc600627fa886cb"} Jun 09 05:33:18.687475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.687379 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:18.689247 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.689220 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="809e3e7325dbd93fc6fdd5c6fd70335c6187ddd2ad850cf9fed6a929a8bd52eb" exitCode=0 Jun 09 05:33:18.689338 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.689313 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"809e3e7325dbd93fc6fdd5c6fd70335c6187ddd2ad850cf9fed6a929a8bd52eb"} Jun 09 05:33:18.693132 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.693112 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" Jun 09 05:33:18.705752 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.705707 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" podStartSLOduration=1.943686909 podStartE2EDuration="6.705689226s" totalImagesPullingTime="4.762002317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:33:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:18.704084595 +0000 UTC m=+272.398427599" watchObservedRunningTime="2026-06-09 05:33:18.705689226 +0000 UTC m=+272.400032211" Jun 09 05:33:18.718899 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:18.718854 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-x592k" podStartSLOduration=1.887038183 podStartE2EDuration="4.718831093s" totalImagesPullingTime="2.83179291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:18.717498566 +0000 UTC m=+272.411841545" watchObservedRunningTime="2026-06-09 05:33:18.718831093 +0000 UTC m=+272.413174063" Jun 09 05:33:22.710878 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:22.710851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"5917e81a5702bde5d3903993d90e9afcf21341a1bbb55d058d01addec7959e23"} Jun 09 05:33:22.711208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:22.710914 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"a9be98dc17ffb0893ab09661f863af65ad30357a3099e9bea8c63cd9f54fdce2"} Jun 09 05:33:23.718232 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:23.718197 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"431790cb226c42f9cd040c6453edf8d06aaeb3194edb861d6db87f2dde8cc525"} Jun 09 05:33:23.718708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:23.718261 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"dfc336b4d816c44ec9bb09c9a5bf10fd68357bf6d03b0f705329fb6f7af14eea"} Jun 09 05:33:23.718708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:23.718288 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"9478cff10da94a08ed890cb931177465bcf47d4deb56df5ca13148c96000dbb0"} Jun 09 05:33:23.718708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:23.718319 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerStarted","Data":"38eeec9023e9cb3960fd088447662985caffcb525c9d844aaf04cbb626ed1a38"} Jun 09 05:33:23.745089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:23.745035 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.90207146 podStartE2EDuration="8.745015246s" totalImagesPullingTime="5.818324864s" totalInitContainerRuntime="24.618922ms" isStatefulPod=false podCreationTimestamp="2026-06-09 05:33:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:23.742907307 +0000 UTC m=+277.437250286" watchObservedRunningTime="2026-06-09 05:33:23.745015246 +0000 UTC m=+277.439358226" Jun 09 05:33:24.305160 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:24.305125 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:24.354579 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:24.354550 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:24.700336 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:24.700308 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b676678c7-l7dkh" Jun 09 05:33:24.923058 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:24.923029 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:24.974751 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:24.974673 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:25.867324 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:25.867291 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:33:27.298287 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:33:27.298248 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8r8dx" podUID="40108820-3325-4e88-a8b1-056266249a9b" Jun 09 05:33:27.298287 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:33:27.298257 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-864f978974-4b5sl" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" Jun 09 05:33:27.735718 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:27.735687 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:33:27.735897 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:27.735689 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r8dx" Jun 09 05:33:31.110531 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.110487 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:31.159600 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.159569 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:31.169852 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.169825 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:33:31.169935 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.169873 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:33:31.172242 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.172225 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40108820-3325-4e88-a8b1-056266249a9b-metrics-tls\") pod \"dns-default-8r8dx\" (UID: \"40108820-3325-4e88-a8b1-056266249a9b\") " pod="openshift-dns/dns-default-8r8dx" Jun 09 05:33:31.172319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.172271 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"image-registry-864f978974-4b5sl\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:33:31.340083 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.340048 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ww7k9\"" Jun 09 05:33:31.340083 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.340048 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2rk2w\"" Jun 09 05:33:31.347019 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.346997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:33:31.347107 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.347090 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8r8dx" Jun 09 05:33:31.516967 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.516938 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864f978974-4b5sl"] Jun 09 05:33:31.519293 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:31.519222 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eabb6ca_08b7_4322_9a7b_ca108ac80a44.slice/crio-fc88d2843a6d58f3b469904da31552fdf48a45cf5ece634131ac1e8d5fd9d1aa WatchSource:0}: Error finding container fc88d2843a6d58f3b469904da31552fdf48a45cf5ece634131ac1e8d5fd9d1aa: Status 404 returned error can't find the container with id fc88d2843a6d58f3b469904da31552fdf48a45cf5ece634131ac1e8d5fd9d1aa Jun 09 05:33:31.541743 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.541720 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8r8dx"] Jun 09 05:33:31.543677 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:33:31.543655 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40108820_3325_4e88_a8b1_056266249a9b.slice/crio-7140048dac51ba501fad31193977239d6d6fe7d18a0e75333e6b6c25d5e8fa97 WatchSource:0}: Error finding container 7140048dac51ba501fad31193977239d6d6fe7d18a0e75333e6b6c25d5e8fa97: Status 404 returned error can't find the container with id 7140048dac51ba501fad31193977239d6d6fe7d18a0e75333e6b6c25d5e8fa97 Jun 09 05:33:31.752041 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.751956 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8r8dx" event={"ID":"40108820-3325-4e88-a8b1-056266249a9b","Type":"ContainerStarted","Data":"7140048dac51ba501fad31193977239d6d6fe7d18a0e75333e6b6c25d5e8fa97"} Jun 09 05:33:31.753554 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.753517 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864f978974-4b5sl" event={"ID":"8eabb6ca-08b7-4322-9a7b-ca108ac80a44","Type":"ContainerStarted","Data":"39ee5c93882fe22bb7a41d978cd256d2d052b7c3ad5b68a03be55d0f36be7c21"} Jun 09 05:33:31.753554 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.753558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864f978974-4b5sl" event={"ID":"8eabb6ca-08b7-4322-9a7b-ca108ac80a44","Type":"ContainerStarted","Data":"fc88d2843a6d58f3b469904da31552fdf48a45cf5ece634131ac1e8d5fd9d1aa"} Jun 09 05:33:31.753713 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.753598 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:33:31.772991 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:31.772932 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-864f978974-4b5sl" podStartSLOduration=284.772912288 podStartE2EDuration="4m44.772912288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:31.770340637 +0000 UTC m=+285.464683616" watchObservedRunningTime="2026-06-09 05:33:31.772912288 +0000 UTC m=+285.467255269" Jun 09 05:33:32.821148 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:32.821111 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:32.869046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:32.869014 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:33.764377 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:33.764340 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8r8dx" event={"ID":"40108820-3325-4e88-a8b1-056266249a9b","Type":"ContainerStarted","Data":"8cdc0c2e2ef1c95c8b2f8d365fd8102f338dc43e7127f66224fe6c141830f7ed"} Jun 09 05:33:34.769900 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:34.769866 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8r8dx" event={"ID":"40108820-3325-4e88-a8b1-056266249a9b","Type":"ContainerStarted","Data":"de954d6f10cdafef42ce4c3e31f234e1c390cd7b6c748a758337bccbb39f8165"} Jun 09 05:33:34.770405 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:34.769972 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8r8dx" Jun 09 05:33:34.785413 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:34.785337 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8r8dx" podStartSLOduration=251.772499106 podStartE2EDuration="4m13.785321679s" totalImagesPullingTime="2.012822573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:33:34.784604659 +0000 UTC m=+288.478947632" watchObservedRunningTime="2026-06-09 05:33:34.785321679 +0000 UTC m=+288.479664681" Jun 09 05:33:42.335335 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:42.335299 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:42.382406 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:42.382358 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:44.775974 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:44.775949 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8r8dx" Jun 09 05:33:46.738691 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:46.738659 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:33:46.739659 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:46.739630 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:33:46.750203 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:46.750182 2567 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 05:33:48.991428 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:48.991375 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:49.958056 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:49.958021 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:51.351599 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:51.351563 2567 patch_prober.go:28] interesting pod/image-registry-864f978974-4b5sl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 05:33:51.351982 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:51.351624 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864f978974-4b5sl" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 05:33:52.011506 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:52.011473 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:52.762059 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:52.762030 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:33:52.948668 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:52.948619 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:53.049868 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:53.049767 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:53.883629 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:53.883595 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:53.960506 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:53.960472 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:54.617011 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:54.616977 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:54.683054 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:54.683024 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864f978974-4b5sl"] Jun 09 05:33:54.687602 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:54.687576 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:54.757774 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:54.757744 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:54.858955 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:54.858917 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:54.962029 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:54.961993 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:55.059428 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:55.059376 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:55.160408 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:55.160349 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:55.260441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:55.260341 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:57.796646 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:57.796614 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:57.849004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:57.848971 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:57.900761 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:57.900734 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:58.102857 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:58.102764 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:33:58.150322 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:33:58.150287 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:01.126098 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:01.126065 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:01.227040 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:01.227005 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:10.034732 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:10.034696 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:10.134991 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:10.134958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:15.867473 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:15.867414 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:15.882780 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:15.882747 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:15.943176 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:15.943147 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:16.489623 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:16.489590 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:16.541542 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:16.541520 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:17.317884 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:17.317854 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:17.418988 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:17.418956 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:17.464878 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:17.464854 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:19.702957 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.702900 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864f978974-4b5sl" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" containerName="registry" containerID="cri-o://39ee5c93882fe22bb7a41d978cd256d2d052b7c3ad5b68a03be55d0f36be7c21" gracePeriod=30 Jun 09 05:34:19.945062 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.945037 2567 generic.go:356] "Generic (PLEG): container finished" podID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" containerID="39ee5c93882fe22bb7a41d978cd256d2d052b7c3ad5b68a03be55d0f36be7c21" exitCode=0 Jun 09 05:34:19.945181 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.945107 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864f978974-4b5sl" event={"ID":"8eabb6ca-08b7-4322-9a7b-ca108ac80a44","Type":"ContainerDied","Data":"39ee5c93882fe22bb7a41d978cd256d2d052b7c3ad5b68a03be55d0f36be7c21"} Jun 09 05:34:19.945181 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.945155 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864f978974-4b5sl" event={"ID":"8eabb6ca-08b7-4322-9a7b-ca108ac80a44","Type":"ContainerDied","Data":"fc88d2843a6d58f3b469904da31552fdf48a45cf5ece634131ac1e8d5fd9d1aa"} Jun 09 05:34:19.945181 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.945170 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc88d2843a6d58f3b469904da31552fdf48a45cf5ece634131ac1e8d5fd9d1aa" Jun 09 05:34:19.945453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.945438 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:34:19.980185 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980100 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-installation-pull-secrets\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980185 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980144 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980403 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980183 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-image-registry-private-configuration\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980403 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980213 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv4r7\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-kube-api-access-nv4r7\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980403 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980274 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-bound-sa-token\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980403 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980298 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-ca-trust-extracted\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980403 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980319 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-certificates\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980403 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980353 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-trusted-ca\") pod \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\" (UID: \"8eabb6ca-08b7-4322-9a7b-ca108ac80a44\") " Jun 09 05:34:19.980998 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.980932 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:19.981533 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.981505 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:19.983004 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.982960 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:19.983207 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.983162 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:19.984553 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.984526 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-kube-api-access-nv4r7" (OuterVolumeSpecName: "kube-api-access-nv4r7") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "kube-api-access-nv4r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:34:19.984832 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.984804 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:34:19.985009 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.984990 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:34:19.989981 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:19.989957 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8eabb6ca-08b7-4322-9a7b-ca108ac80a44" (UID: "8eabb6ca-08b7-4322-9a7b-ca108ac80a44"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 05:34:20.081676 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081632 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-tls\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081676 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081671 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-image-registry-private-configuration\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081676 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081685 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv4r7\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-kube-api-access-nv4r7\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081698 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-bound-sa-token\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081709 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-ca-trust-extracted\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081720 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-registry-certificates\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081730 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-trusted-ca\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.081922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.081742 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8eabb6ca-08b7-4322-9a7b-ca108ac80a44-installation-pull-secrets\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:20.948099 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.948070 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864f978974-4b5sl" Jun 09 05:34:20.964636 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.964601 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864f978974-4b5sl"] Jun 09 05:34:20.967576 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:20.967551 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-864f978974-4b5sl"] Jun 09 05:34:22.235031 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:22.235002 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:22.283921 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:22.283883 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:22.798999 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:22.798966 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" path="/var/lib/kubelet/pods/8eabb6ca-08b7-4322-9a7b-ca108ac80a44/volumes" Jun 09 05:34:25.038440 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:25.038407 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:25.094945 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:25.094918 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:25.386062 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:25.386028 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:25.434221 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:25.434193 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:27.999651 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:27.999618 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:28.049588 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:28.049556 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:33.848907 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.848872 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:34:33.849315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.849156 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="prometheus" containerID="cri-o://a9be98dc17ffb0893ab09661f863af65ad30357a3099e9bea8c63cd9f54fdce2" gracePeriod=600 Jun 09 05:34:33.849315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.849176 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy" containerID="cri-o://9478cff10da94a08ed890cb931177465bcf47d4deb56df5ca13148c96000dbb0" gracePeriod=600 Jun 09 05:34:33.849315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.849194 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="thanos-sidecar" containerID="cri-o://431790cb226c42f9cd040c6453edf8d06aaeb3194edb861d6db87f2dde8cc525" gracePeriod=600 Jun 09 05:34:33.849315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.849245 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="config-reloader" containerID="cri-o://5917e81a5702bde5d3903993d90e9afcf21341a1bbb55d058d01addec7959e23" gracePeriod=600 Jun 09 05:34:33.849315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.849257 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-thanos" containerID="cri-o://dfc336b4d816c44ec9bb09c9a5bf10fd68357bf6d03b0f705329fb6f7af14eea" gracePeriod=600 Jun 09 05:34:33.849591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:33.849254 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-web" containerID="cri-o://38eeec9023e9cb3960fd088447662985caffcb525c9d844aaf04cbb626ed1a38" gracePeriod=600 Jun 09 05:34:34.007048 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007012 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="dfc336b4d816c44ec9bb09c9a5bf10fd68357bf6d03b0f705329fb6f7af14eea" exitCode=0 Jun 09 05:34:34.007048 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007037 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="9478cff10da94a08ed890cb931177465bcf47d4deb56df5ca13148c96000dbb0" exitCode=0 Jun 09 05:34:34.007048 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007048 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="38eeec9023e9cb3960fd088447662985caffcb525c9d844aaf04cbb626ed1a38" exitCode=0 Jun 09 05:34:34.007048 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007055 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="431790cb226c42f9cd040c6453edf8d06aaeb3194edb861d6db87f2dde8cc525" exitCode=0 Jun 09 05:34:34.007048 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007061 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="5917e81a5702bde5d3903993d90e9afcf21341a1bbb55d058d01addec7959e23" exitCode=0 Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007068 2567 generic.go:356] "Generic (PLEG): container finished" podID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerID="a9be98dc17ffb0893ab09661f863af65ad30357a3099e9bea8c63cd9f54fdce2" exitCode=0 Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007077 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"dfc336b4d816c44ec9bb09c9a5bf10fd68357bf6d03b0f705329fb6f7af14eea"} Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007139 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"9478cff10da94a08ed890cb931177465bcf47d4deb56df5ca13148c96000dbb0"} Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007166 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"38eeec9023e9cb3960fd088447662985caffcb525c9d844aaf04cbb626ed1a38"} Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007191 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"431790cb226c42f9cd040c6453edf8d06aaeb3194edb861d6db87f2dde8cc525"} Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007216 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"5917e81a5702bde5d3903993d90e9afcf21341a1bbb55d058d01addec7959e23"} Jun 09 05:34:34.007452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.007241 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"a9be98dc17ffb0893ab09661f863af65ad30357a3099e9bea8c63cd9f54fdce2"} Jun 09 05:34:34.083804 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.083781 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:34.192757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.192726 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-config-out\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.192757 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.192761 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-kube-rbac-proxy\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.192783 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-config\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.192803 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsnvf\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-kube-api-access-zsnvf\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.192835 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-metrics-client-certs\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.192973 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-db\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193046 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193035 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-2\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193067 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-thanos-prometheus-http-client-file\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193095 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-tls-assets\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193136 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-grpc-tls\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-web-config\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193191 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-trusted-ca-bundle\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193216 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193246 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-1\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193277 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-0\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193315 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-metrics-client-ca\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193339 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193368 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-tls\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193425 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-serving-certs-ca-bundle\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193464 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-kubelet-serving-ca-bundle\") pod \"bea6cd74-bd49-44e8-abef-632333fcdb20\" (UID: \"bea6cd74-bd49-44e8-abef-632333fcdb20\") " Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193478 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.193734 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.193691 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.194089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.194005 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.194307 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.194280 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 05:34:34.194374 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.194315 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.195663 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.195617 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.195663 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.195632 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.195663 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.195635 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-kube-api-access-zsnvf" (OuterVolumeSpecName: "kube-api-access-zsnvf") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "kube-api-access-zsnvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:34:34.195990 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.195666 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-config" (OuterVolumeSpecName: "config") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.195990 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.195815 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.195990 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.195980 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.196402 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.196362 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.196983 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.196866 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.196983 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.196907 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-config-out" (OuterVolumeSpecName: "config-out") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 05:34:34.197229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.197025 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.197229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.197195 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.197789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.197769 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 05:34:34.197865 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.197851 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:34:34.198354 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.198333 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.198457 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.198427 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.207532 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.207507 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-web-config" (OuterVolumeSpecName: "web-config") pod "bea6cd74-bd49-44e8-abef-632333fcdb20" (UID: "bea6cd74-bd49-44e8-abef-632333fcdb20"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:34:34.294541 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294502 2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-metrics-client-certs\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294541 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294539 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-db\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294548 2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294559 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-tls-assets\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294568 2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-grpc-tls\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294576 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-web-config\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294584 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294593 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294601 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294610 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294618 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-metrics-client-ca\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294628 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294636 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294644 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294655 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bea6cd74-bd49-44e8-abef-632333fcdb20-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294664 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bea6cd74-bd49-44e8-abef-632333fcdb20-config-out\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294672 2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-secret-kube-rbac-proxy\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294682 2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bea6cd74-bd49-44e8-abef-632333fcdb20-config\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:34.294701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:34.294689 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsnvf\" (UniqueName: \"kubernetes.io/projected/bea6cd74-bd49-44e8-abef-632333fcdb20-kube-api-access-zsnvf\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:34:35.013266 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.013230 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bea6cd74-bd49-44e8-abef-632333fcdb20","Type":"ContainerDied","Data":"a15aa0c814f7326f759ca5263cee869e867f6fdb89a581bef440ce8c305f75ba"} Jun 09 05:34:35.013750 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.013307 2567 scope.go:117] "RemoveContainer" containerID="dfc336b4d816c44ec9bb09c9a5bf10fd68357bf6d03b0f705329fb6f7af14eea" Jun 09 05:34:35.013750 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.013323 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.020985 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.020965 2567 scope.go:117] "RemoveContainer" containerID="9478cff10da94a08ed890cb931177465bcf47d4deb56df5ca13148c96000dbb0" Jun 09 05:34:35.027521 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.027504 2567 scope.go:117] "RemoveContainer" containerID="38eeec9023e9cb3960fd088447662985caffcb525c9d844aaf04cbb626ed1a38" Jun 09 05:34:35.033787 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.033772 2567 scope.go:117] "RemoveContainer" containerID="431790cb226c42f9cd040c6453edf8d06aaeb3194edb861d6db87f2dde8cc525" Jun 09 05:34:35.040342 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.040326 2567 scope.go:117] "RemoveContainer" containerID="5917e81a5702bde5d3903993d90e9afcf21341a1bbb55d058d01addec7959e23" Jun 09 05:34:35.046571 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.046555 2567 scope.go:117] "RemoveContainer" containerID="a9be98dc17ffb0893ab09661f863af65ad30357a3099e9bea8c63cd9f54fdce2" Jun 09 05:34:35.049990 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.049965 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:34:35.054289 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.054271 2567 scope.go:117] "RemoveContainer" containerID="809e3e7325dbd93fc6fdd5c6fd70335c6187ddd2ad850cf9fed6a929a8bd52eb" Jun 09 05:34:35.054528 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.054475 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:34:35.063208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.063187 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:35.074041 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.073995 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:34:35.074290 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074278 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="prometheus" Jun 09 05:34:35.074329 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074292 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="prometheus" Jun 09 05:34:35.074329 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074300 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" containerName="registry" Jun 09 05:34:35.074329 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074327 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" containerName="registry" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074344 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="thanos-sidecar" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074350 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="thanos-sidecar" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074356 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-web" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074362 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-web" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074371 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-thanos" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074377 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-thanos" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074403 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="config-reloader" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074411 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="config-reloader" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074418 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074423 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074429 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="init-config-reloader" Jun 09 05:34:35.074439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074434 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="init-config-reloader" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074487 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="thanos-sidecar" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074494 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-web" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074500 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy-thanos" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074505 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eabb6ca-08b7-4322-9a7b-ca108ac80a44" containerName="registry" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074510 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="kube-rbac-proxy" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074517 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="prometheus" Jun 09 05:34:35.074775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.074528 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" containerName="config-reloader" Jun 09 05:34:35.077913 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.077898 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.080451 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080431 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-f7hb2\"" Jun 09 05:34:35.080552 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080504 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 05:34:35.080552 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080517 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 05:34:35.080657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080584 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 05:34:35.080657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080611 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-301lng8agc3ov\"" Jun 09 05:34:35.080657 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080646 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 05:34:35.080816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080701 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 05:34:35.080939 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080923 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 05:34:35.081001 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080942 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 05:34:35.081001 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.080953 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 05:34:35.081229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.081214 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 05:34:35.081462 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.081446 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 05:34:35.081541 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.081500 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 05:34:35.081680 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.081667 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 05:34:35.081969 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.081954 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 05:34:35.084092 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.084071 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 05:34:35.086008 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.085992 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 05:34:35.097045 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.097026 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:34:35.128381 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.128356 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:35.201237 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201203 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201237 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201241 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201258 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6btxq\" (UniqueName: \"kubernetes.io/projected/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-kube-api-access-6btxq\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201275 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-config\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201296 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201461 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201532 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201598 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201641 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201664 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201684 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.201715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201715 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.202011 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.202011 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.202011 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.201776 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302578 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302538 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302578 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302584 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302631 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302653 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302938 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302823 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302938 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302873 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.302938 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302906 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302942 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.302993 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303060 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303082 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303135 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6btxq\" (UniqueName: \"kubernetes.io/projected/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-kube-api-access-6btxq\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303184 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-config\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303223 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303248 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303567 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303655 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303671 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.303865 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.303842 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.304104 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.304083 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.305127 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.304739 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.306059 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.305752 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.306059 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.305982 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.306782 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.306302 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.306782 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.306598 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-config\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.306782 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.306680 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.306984 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.306853 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.307488 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.307463 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.307770 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.307740 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.308123 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.308097 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.308463 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.308437 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.308560 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.308544 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.308741 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.308722 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.309086 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.309065 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.310890 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.310868 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6btxq\" (UniqueName: \"kubernetes.io/projected/3e83ed02-1542-4b4e-9af8-fe32c2f8922e-kube-api-access-6btxq\") pod \"prometheus-k8s-0\" (UID: \"3e83ed02-1542-4b4e-9af8-fe32c2f8922e\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.387609 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.387571 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:34:35.518450 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:35.518421 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 05:34:35.521337 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:34:35.521313 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e83ed02_1542_4b4e_9af8_fe32c2f8922e.slice/crio-e70d6c7dadb7ac663080726d0d816aad56d1481e472675d42e4496a9dd1d1171 WatchSource:0}: Error finding container e70d6c7dadb7ac663080726d0d816aad56d1481e472675d42e4496a9dd1d1171: Status 404 returned error can't find the container with id e70d6c7dadb7ac663080726d0d816aad56d1481e472675d42e4496a9dd1d1171 Jun 09 05:34:36.017945 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:36.017911 2567 generic.go:356] "Generic (PLEG): container finished" podID="3e83ed02-1542-4b4e-9af8-fe32c2f8922e" containerID="ae55b6003f2d863d42a5f7d497525daea883f3aff6eea7687dba778925489424" exitCode=0 Jun 09 05:34:36.018416 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:36.018004 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerDied","Data":"ae55b6003f2d863d42a5f7d497525daea883f3aff6eea7687dba778925489424"} Jun 09 05:34:36.018416 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:36.018071 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"e70d6c7dadb7ac663080726d0d816aad56d1481e472675d42e4496a9dd1d1171"} Jun 09 05:34:36.798622 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:36.798583 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bea6cd74-bd49-44e8-abef-632333fcdb20" path="/var/lib/kubelet/pods/bea6cd74-bd49-44e8-abef-632333fcdb20/volumes" Jun 09 05:34:37.026069 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.026035 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"9eb71aef41a77d0c395eacc532cbb2139b1d9d56757cc0bd851e81a0254b918f"} Jun 09 05:34:37.026453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.026113 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"a9f8f2c1c2eca3aa14c91cb2a592eea2e8a80f4c2083f18cc14aebe9b2ae139d"} Jun 09 05:34:37.026453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.026158 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"2a8590d0549f1d3a26eb7c40c5a3e13ab2936562b83f53a9cf8748d467a9af42"} Jun 09 05:34:37.026453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.026197 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"0959143d6c041bada8b607f60ca4913f37f030e010a25735a40702e3f5e47a0a"} Jun 09 05:34:37.026453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.026231 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"f6936b353122e47b3049c669e64f6fc36c5a379aa51bdf1eb4c5fbcdcaa1a7f6"} Jun 09 05:34:37.026453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.026267 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e83ed02-1542-4b4e-9af8-fe32c2f8922e","Type":"ContainerStarted","Data":"27d26a519c497a27603f4caaceb4ea8c613449c8da90455039fe990baf5909b6"} Jun 09 05:34:37.054456 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:37.054327 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.035900629 podStartE2EDuration="2.054307834s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.407205ms" isStatefulPod=false podCreationTimestamp="2026-06-09 05:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:34:37.052996571 +0000 UTC m=+350.747339586" watchObservedRunningTime="2026-06-09 05:34:37.054307834 +0000 UTC m=+350.748650815" Jun 09 05:34:38.702342 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:38.702313 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:38.805411 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:38.805352 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:38.852802 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:38.852772 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:40.048204 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:40.048170 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:40.148583 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:40.148540 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:34:40.387863 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:34:40.387832 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:35:11.221103 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:11.221065 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:11.322166 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:11.322139 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:35.388717 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:35.388673 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:35:35.403704 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:35.403678 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:35:36.263093 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:36.263066 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 05:35:37.422591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:37.422558 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:37.523761 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:37.523733 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:37.622318 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:37.622288 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:38.854013 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:38.853978 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:41.091198 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:41.091157 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:41.148010 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:41.147973 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:43.003970 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:43.003937 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:43.105572 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:43.105536 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:43.380057 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:43.380024 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:43.477943 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:43.477912 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:46.317868 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:46.317822 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:46.916434 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:46.916397 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:48.086172 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:48.086137 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:48.136141 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:48.136116 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:50.786422 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:50.786376 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:54.700236 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:54.700189 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:35:54.798854 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:35:54.798809 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:00.730273 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:00.730230 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:00.778955 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:00.778926 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:02.472737 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:02.472697 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:15.305235 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:15.305149 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:15.399930 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:15.399885 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:15.948246 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:15.948213 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.132425 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.132371 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.231808 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.231727 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.275851 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.275823 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2"] Jun 09 05:36:16.278830 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.278810 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.281617 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.281594 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 05:36:16.281726 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.281700 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 05:36:16.281819 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.281804 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mjks9\"" Jun 09 05:36:16.282013 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.281993 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 05:36:16.282066 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.282016 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 05:36:16.296684 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.296663 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2"] Jun 09 05:36:16.379287 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.379248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.393158 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.393137 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce1caaf1-a826-4e7a-8710-5467bcdd255f-tmp\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.393280 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.393233 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njkk2\" (UniqueName: \"kubernetes.io/projected/ce1caaf1-a826-4e7a-8710-5467bcdd255f-kube-api-access-njkk2\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.393280 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.393272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1caaf1-a826-4e7a-8710-5467bcdd255f-webhook-cert\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.393355 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.393304 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce1caaf1-a826-4e7a-8710-5467bcdd255f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.393355 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.393346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce1caaf1-a826-4e7a-8710-5467bcdd255f-manifests\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.479324 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.479295 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.494534 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.494476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce1caaf1-a826-4e7a-8710-5467bcdd255f-manifests\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.494534 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.494508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce1caaf1-a826-4e7a-8710-5467bcdd255f-tmp\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.494647 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.494568 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njkk2\" (UniqueName: \"kubernetes.io/projected/ce1caaf1-a826-4e7a-8710-5467bcdd255f-kube-api-access-njkk2\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.494647 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.494595 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1caaf1-a826-4e7a-8710-5467bcdd255f-webhook-cert\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.494647 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.494629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce1caaf1-a826-4e7a-8710-5467bcdd255f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.494932 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.494908 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce1caaf1-a826-4e7a-8710-5467bcdd255f-manifests\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.496968 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.496938 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce1caaf1-a826-4e7a-8710-5467bcdd255f-tmp\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.497063 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.497005 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce1caaf1-a826-4e7a-8710-5467bcdd255f-webhook-cert\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.497157 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.497137 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce1caaf1-a826-4e7a-8710-5467bcdd255f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.503847 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.503827 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njkk2\" (UniqueName: \"kubernetes.io/projected/ce1caaf1-a826-4e7a-8710-5467bcdd255f-kube-api-access-njkk2\") pod \"opendatahub-operator-controller-manager-66ccd5c574-zmwk2\" (UID: \"ce1caaf1-a826-4e7a-8710-5467bcdd255f\") " pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.579563 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.579527 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.589488 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.589451 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:16.680658 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.680615 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:16.720479 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.720438 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2"] Jun 09 05:36:16.722718 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:36:16.722683 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1caaf1_a826_4e7a_8710_5467bcdd255f.slice/crio-4cfa8b50b590a110b569d7db3df5d24ce4a0a57aa1db3561cf5d59e560d26b01 WatchSource:0}: Error finding container 4cfa8b50b590a110b569d7db3df5d24ce4a0a57aa1db3561cf5d59e560d26b01: Status 404 returned error can't find the container with id 4cfa8b50b590a110b569d7db3df5d24ce4a0a57aa1db3561cf5d59e560d26b01 Jun 09 05:36:16.724512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.724496 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 05:36:16.780278 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:16.780217 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:17.402538 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:17.402496 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" event={"ID":"ce1caaf1-a826-4e7a-8710-5467bcdd255f","Type":"ContainerStarted","Data":"4cfa8b50b590a110b569d7db3df5d24ce4a0a57aa1db3561cf5d59e560d26b01"} Jun 09 05:36:20.415596 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:20.415563 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" event={"ID":"ce1caaf1-a826-4e7a-8710-5467bcdd255f","Type":"ContainerStarted","Data":"abcb3da493fade2a13524223a56e38a79b6ded3e779adf994ad4bf71f4f8728c"} Jun 09 05:36:21.420447 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:21.420408 2567 generic.go:356] "Generic (PLEG): container finished" podID="ce1caaf1-a826-4e7a-8710-5467bcdd255f" containerID="abcb3da493fade2a13524223a56e38a79b6ded3e779adf994ad4bf71f4f8728c" exitCode=0 Jun 09 05:36:21.420864 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:21.420493 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" event={"ID":"ce1caaf1-a826-4e7a-8710-5467bcdd255f","Type":"ContainerDied","Data":"abcb3da493fade2a13524223a56e38a79b6ded3e779adf994ad4bf71f4f8728c"} Jun 09 05:36:22.426650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:22.426616 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" event={"ID":"ce1caaf1-a826-4e7a-8710-5467bcdd255f","Type":"ContainerStarted","Data":"dc98f47c59dd4758fb975114ece56427f3174e13825cebd0095b94561a7bc449"} Jun 09 05:36:22.427072 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:22.426761 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:22.451226 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:22.451153 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" podStartSLOduration=2.478842884 podStartE2EDuration="6.451139359s" totalImagesPullingTime="3.065638005s" totalInitContainerRuntime="906.65847ms" isStatefulPod=false podCreationTimestamp="2026-06-09 05:36:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:36:22.448715669 +0000 UTC m=+456.143058658" watchObservedRunningTime="2026-06-09 05:36:22.451139359 +0000 UTC m=+456.145482338" Jun 09 05:36:23.370789 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:23.370753 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:23.426251 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:23.426222 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:26.865969 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:26.865939 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:26.970779 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:26.970746 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:33.432276 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:33.432239 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66ccd5c574-zmwk2" Jun 09 05:36:34.872221 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:34.872186 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:35.004299 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:35.004265 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:35.525007 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:35.524978 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:35.623894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:35.623861 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:35.725002 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:35.724971 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:36.037711 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:36.037671 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:36.101659 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:36.101626 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:36.495509 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:36.495474 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:40.694575 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.694541 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:40.756444 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.756406 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4"] Jun 09 05:36:40.759708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.759686 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.762033 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.762013 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 09 05:36:40.762033 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.762022 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 09 05:36:40.762445 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.762422 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:40.762961 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.762945 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 09 05:36:40.763018 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.762958 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 09 05:36:40.763078 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.763059 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nmfrv\"" Jun 09 05:36:40.763174 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.763138 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 09 05:36:40.765491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.765465 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qs4t\" (UniqueName: \"kubernetes.io/projected/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-kube-api-access-8qs4t\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.765593 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.765538 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-manager-config\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.765652 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.765599 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-metrics-cert\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.765704 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.765670 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-cert\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.781080 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.781052 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4"] Jun 09 05:36:40.866644 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.866616 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-manager-config\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.866791 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.866689 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-metrics-cert\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.866791 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.866740 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-cert\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.866791 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.866775 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qs4t\" (UniqueName: \"kubernetes.io/projected/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-kube-api-access-8qs4t\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.867321 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.867299 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-manager-config\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.869291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.869261 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-cert\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.869291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.869285 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-metrics-cert\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:40.878831 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:40.878805 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qs4t\" (UniqueName: \"kubernetes.io/projected/bbc854c2-f5dd-4efa-b7ef-2e377c2a7072-kube-api-access-8qs4t\") pod \"lws-controller-manager-55f7df5cdd-qk9g4\" (UID: \"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072\") " pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:41.069823 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:41.069744 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:41.199522 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:41.199496 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4"] Jun 09 05:36:41.201831 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:36:41.201805 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc854c2_f5dd_4efa_b7ef_2e377c2a7072.slice/crio-dd32b6b5c836693abc8fcca9888b15b5d438127edb2a5571428bf590369746f4 WatchSource:0}: Error finding container dd32b6b5c836693abc8fcca9888b15b5d438127edb2a5571428bf590369746f4: Status 404 returned error can't find the container with id dd32b6b5c836693abc8fcca9888b15b5d438127edb2a5571428bf590369746f4 Jun 09 05:36:41.502108 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:41.502064 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" event={"ID":"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072","Type":"ContainerStarted","Data":"dd32b6b5c836693abc8fcca9888b15b5d438127edb2a5571428bf590369746f4"} Jun 09 05:36:44.517560 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:44.517517 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" event={"ID":"bbc854c2-f5dd-4efa-b7ef-2e377c2a7072","Type":"ContainerStarted","Data":"2c35e5f125690234057f861ecbdcde1a45826c33b567e6166763cccb428837c7"} Jun 09 05:36:44.517943 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:44.517665 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:36:44.540688 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:44.540642 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" podStartSLOduration=2.104810569 podStartE2EDuration="4.540629556s" totalImagesPullingTime="2.435818987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:36:44.536162465 +0000 UTC m=+478.230505444" watchObservedRunningTime="2026-06-09 05:36:44.540629556 +0000 UTC m=+478.234972535" Jun 09 05:36:45.486104 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.486065 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:45.578438 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.578396 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:45.843913 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.843830 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:45.914986 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.914949 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g"] Jun 09 05:36:45.918817 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.918797 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:45.921436 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.921411 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hvf7g\"" Jun 09 05:36:45.921656 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.921638 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 09 05:36:45.922302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.922288 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 09 05:36:45.933426 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.933379 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g"] Jun 09 05:36:45.937458 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:45.937433 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:46.013259 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.013221 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ac14b4b-1843-4403-a3e0-64a6a620578c-tls-certs\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.013450 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.013283 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ac14b4b-1843-4403-a3e0-64a6a620578c-tmp\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.013450 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.013320 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6tzp\" (UniqueName: \"kubernetes.io/projected/9ac14b4b-1843-4403-a3e0-64a6a620578c-kube-api-access-g6tzp\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.114834 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.114751 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ac14b4b-1843-4403-a3e0-64a6a620578c-tls-certs\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.114834 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.114804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ac14b4b-1843-4403-a3e0-64a6a620578c-tmp\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.114834 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.114826 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6tzp\" (UniqueName: \"kubernetes.io/projected/9ac14b4b-1843-4403-a3e0-64a6a620578c-kube-api-access-g6tzp\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.115147 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:36:46.114917 2567 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 09 05:36:46.115147 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:36:46.114992 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac14b4b-1843-4403-a3e0-64a6a620578c-tls-certs podName:9ac14b4b-1843-4403-a3e0-64a6a620578c nodeName:}" failed. No retries permitted until 2026-06-09 05:36:46.614971392 +0000 UTC m=+480.309314353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9ac14b4b-1843-4403-a3e0-64a6a620578c-tls-certs") pod "kube-auth-proxy-85cbb6b997-hm92g" (UID: "9ac14b4b-1843-4403-a3e0-64a6a620578c") : secret "kube-auth-proxy-tls" not found Jun 09 05:36:46.117015 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.116993 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ac14b4b-1843-4403-a3e0-64a6a620578c-tmp\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.125708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.125682 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6tzp\" (UniqueName: \"kubernetes.io/projected/9ac14b4b-1843-4403-a3e0-64a6a620578c-kube-api-access-g6tzp\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.295190 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.295158 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:46.620146 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.620113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ac14b4b-1843-4403-a3e0-64a6a620578c-tls-certs\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.622355 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.622333 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ac14b4b-1843-4403-a3e0-64a6a620578c-tls-certs\") pod \"kube-auth-proxy-85cbb6b997-hm92g\" (UID: \"9ac14b4b-1843-4403-a3e0-64a6a620578c\") " pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.660443 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.660411 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:46.785127 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.785090 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:46.831152 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.831119 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hvf7g\"" Jun 09 05:36:46.839009 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.838990 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" Jun 09 05:36:46.882864 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.882766 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:46.938096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.938051 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:46.973818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:46.971715 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g"] Jun 09 05:36:46.976605 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:36:46.976577 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac14b4b_1843_4403_a3e0_64a6a620578c.slice/crio-500c27a9c2b78c88e6cd4e92c1ac34b4011292078454e7cffb59ceaf19abe279 WatchSource:0}: Error finding container 500c27a9c2b78c88e6cd4e92c1ac34b4011292078454e7cffb59ceaf19abe279: Status 404 returned error can't find the container with id 500c27a9c2b78c88e6cd4e92c1ac34b4011292078454e7cffb59ceaf19abe279 Jun 09 05:36:47.532647 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:47.532599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" event={"ID":"9ac14b4b-1843-4403-a3e0-64a6a620578c","Type":"ContainerStarted","Data":"500c27a9c2b78c88e6cd4e92c1ac34b4011292078454e7cffb59ceaf19abe279"} Jun 09 05:36:48.977443 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:48.977362 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:49.079270 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:49.079231 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:36:49.833135 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:49.833112 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 09 05:36:50.548815 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:50.548776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" event={"ID":"9ac14b4b-1843-4403-a3e0-64a6a620578c","Type":"ContainerStarted","Data":"8cd5f257a94c67dc0ccff79d70ba6f8802b2a65de20edad6488077f0ff6bcccb"} Jun 09 05:36:50.564449 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:50.564378 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85cbb6b997-hm92g" podStartSLOduration=2.712188142 podStartE2EDuration="5.56436278s" totalImagesPullingTime="2.852174638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:36:50.563104087 +0000 UTC m=+484.257447066" watchObservedRunningTime="2026-06-09 05:36:50.56436278 +0000 UTC m=+484.258705759" Jun 09 05:36:55.525105 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:36:55.525074 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55f7df5cdd-qk9g4" Jun 09 05:37:00.092687 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:00.092655 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:00.194786 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:00.194743 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:01.195207 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:01.195151 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:02.226691 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:02.226657 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:02.316766 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:02.316732 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:02.474510 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:02.474471 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:04.265820 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:04.265777 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:14.642409 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:14.642344 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:14.709570 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:14.709535 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:14.769929 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:14.769891 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:15.235667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:15.235632 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:15.492738 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:15.492661 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:18.020394 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:18.020350 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:18.118544 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:18.118490 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:18.305438 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:18.305349 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:18.507692 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:18.507656 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:18.610535 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:18.610444 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:18.918920 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:18.918892 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:36.407819 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:36.407729 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:36.552288 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:36.552257 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:36.640324 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:36.640263 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:52.584782 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:52.584740 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:37:52.632143 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:37:52.632106 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:05.141316 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:05.141280 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:09.377437 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:09.377403 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:10.738208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:10.738175 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:12.341805 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:12.341773 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:12.737658 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:12.737623 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:12.988527 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:12.988449 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:15.942093 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:15.942059 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:16.617706 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:16.617659 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:16.736000 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:16.735964 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:17.736217 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.736177 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd"] Jun 09 05:38:17.739438 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.739421 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:17.742030 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.742011 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 05:38:17.742838 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.742821 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 05:38:17.742930 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.742858 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4q97q\"" Jun 09 05:38:17.748462 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.748437 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd"] Jun 09 05:38:17.880088 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.880055 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbq28\" (UniqueName: \"kubernetes.io/projected/6517576c-1c63-44d0-ad4b-d7a7e6c02fce-kube-api-access-hbq28\") pod \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" (UID: \"6517576c-1c63-44d0-ad4b-d7a7e6c02fce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:17.980507 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.980474 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbq28\" (UniqueName: \"kubernetes.io/projected/6517576c-1c63-44d0-ad4b-d7a7e6c02fce-kube-api-access-hbq28\") pod \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" (UID: \"6517576c-1c63-44d0-ad4b-d7a7e6c02fce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:17.989554 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:17.989492 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbq28\" (UniqueName: \"kubernetes.io/projected/6517576c-1c63-44d0-ad4b-d7a7e6c02fce-kube-api-access-hbq28\") pod \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" (UID: \"6517576c-1c63-44d0-ad4b-d7a7e6c02fce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:18.049526 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.049493 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:18.175114 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.175089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd"] Jun 09 05:38:18.177511 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:38:18.177484 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6517576c_1c63_44d0_ad4b_d7a7e6c02fce.slice/crio-24ad18d1ae80a3031991d2162f64dcf21ba1d9747c87ac292bf8b156b9e5bf90 WatchSource:0}: Error finding container 24ad18d1ae80a3031991d2162f64dcf21ba1d9747c87ac292bf8b156b9e5bf90: Status 404 returned error can't find the container with id 24ad18d1ae80a3031991d2162f64dcf21ba1d9747c87ac292bf8b156b9e5bf90 Jun 09 05:38:18.388208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.388176 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:18.486766 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.486732 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:18.589523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.589494 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:18.689229 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.688904 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:18.910816 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:18.910782 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" event={"ID":"6517576c-1c63-44d0-ad4b-d7a7e6c02fce","Type":"ContainerStarted","Data":"24ad18d1ae80a3031991d2162f64dcf21ba1d9747c87ac292bf8b156b9e5bf90"} Jun 09 05:38:19.128840 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:19.128808 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:19.233499 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:19.233469 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:19.386909 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:19.386821 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:20.144679 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:20.144645 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:20.246619 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:20.246573 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:20.921238 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:20.921206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" event={"ID":"6517576c-1c63-44d0-ad4b-d7a7e6c02fce","Type":"ContainerStarted","Data":"5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b"} Jun 09 05:38:20.921444 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:20.921313 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:20.937822 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:20.937774 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" podStartSLOduration=1.949196621 podStartE2EDuration="3.937758108s" totalImagesPullingTime="1.988561487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:38:20.937622724 +0000 UTC m=+574.631965703" watchObservedRunningTime="2026-06-09 05:38:20.937758108 +0000 UTC m=+574.632101088" Jun 09 05:38:21.413338 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.413304 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc"] Jun 09 05:38:21.417085 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.417064 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.418708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.418683 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:21.419274 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.419256 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kcv5g\"" Jun 09 05:38:21.419358 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.419294 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 09 05:38:21.419358 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.419305 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 09 05:38:21.422715 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.422689 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc"] Jun 09 05:38:21.509076 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.509044 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fkt2\" (UniqueName: \"kubernetes.io/projected/826c9a71-f354-4554-bbdb-d561b811b26c-kube-api-access-8fkt2\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.509076 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.509083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/826c9a71-f354-4554-bbdb-d561b811b26c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.509296 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.509107 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/826c9a71-f354-4554-bbdb-d561b811b26c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.556540 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.556507 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:21.609922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.609891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/826c9a71-f354-4554-bbdb-d561b811b26c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.610116 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.609973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fkt2\" (UniqueName: \"kubernetes.io/projected/826c9a71-f354-4554-bbdb-d561b811b26c-kube-api-access-8fkt2\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.610116 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.609997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/826c9a71-f354-4554-bbdb-d561b811b26c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.610721 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.610700 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/826c9a71-f354-4554-bbdb-d561b811b26c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.612295 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.612273 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/826c9a71-f354-4554-bbdb-d561b811b26c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.617617 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.617597 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fkt2\" (UniqueName: \"kubernetes.io/projected/826c9a71-f354-4554-bbdb-d561b811b26c-kube-api-access-8fkt2\") pod \"kuadrant-console-plugin-66b5774985-v6krc\" (UID: \"826c9a71-f354-4554-bbdb-d561b811b26c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.737548 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.737472 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" Jun 09 05:38:21.861596 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.861572 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc"] Jun 09 05:38:21.863490 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:38:21.863463 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod826c9a71_f354_4554_bbdb_d561b811b26c.slice/crio-5ab1bc888a5025ca2e902b0008a139dfe58be2dad4cf983a29341b4dbee05d70 WatchSource:0}: Error finding container 5ab1bc888a5025ca2e902b0008a139dfe58be2dad4cf983a29341b4dbee05d70: Status 404 returned error can't find the container with id 5ab1bc888a5025ca2e902b0008a139dfe58be2dad4cf983a29341b4dbee05d70 Jun 09 05:38:21.928194 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:21.928159 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" event={"ID":"826c9a71-f354-4554-bbdb-d561b811b26c","Type":"ContainerStarted","Data":"5ab1bc888a5025ca2e902b0008a139dfe58be2dad4cf983a29341b4dbee05d70"} Jun 09 05:38:27.751089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:27.751038 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:27.850688 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:27.850644 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:28.199240 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:28.199150 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:28.250400 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:28.250350 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:29.008224 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:29.008180 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:29.067297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:29.067263 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:31.530377 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.530271 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd"] Jun 09 05:38:31.530844 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.530571 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" containerName="manager" containerID="cri-o://5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b" gracePeriod=2 Jun 09 05:38:31.532560 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.532533 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:31.547780 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.547754 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd"] Jun 09 05:38:31.558216 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:31.558183 2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6517576c-1c63-44d0-ad4b-d7a7e6c02fce\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-m9jkd\": pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:31.560271 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:31.560229 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:31.573319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.572640 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q"] Jun 09 05:38:31.573319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.573116 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" containerName="manager" Jun 09 05:38:31.573319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.573136 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" containerName="manager" Jun 09 05:38:31.573319 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.573253 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" containerName="manager" Jun 09 05:38:31.578569 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.578542 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:31.587926 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.587334 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q"] Jun 09 05:38:31.602104 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:31.602066 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:31.700430 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.700395 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr88p\" (UniqueName: \"kubernetes.io/projected/f428f7c2-d4e8-422a-b08b-b883dccfb7f1-kube-api-access-tr88p\") pod \"limitador-operator-controller-manager-859d7b78bd-d2s2q\" (UID: \"f428f7c2-d4e8-422a-b08b-b883dccfb7f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:31.801821 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.801742 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr88p\" (UniqueName: \"kubernetes.io/projected/f428f7c2-d4e8-422a-b08b-b883dccfb7f1-kube-api-access-tr88p\") pod \"limitador-operator-controller-manager-859d7b78bd-d2s2q\" (UID: \"f428f7c2-d4e8-422a-b08b-b883dccfb7f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:31.811067 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.811032 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr88p\" (UniqueName: \"kubernetes.io/projected/f428f7c2-d4e8-422a-b08b-b883dccfb7f1-kube-api-access-tr88p\") pod \"limitador-operator-controller-manager-859d7b78bd-d2s2q\" (UID: \"f428f7c2-d4e8-422a-b08b-b883dccfb7f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:31.917393 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:31.917360 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:33.593741 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:33.593700 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:33.664743 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:33.664711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:36.554499 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:36.554450 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:36.655002 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:36.654968 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:40.352731 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:40.352692 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:40.410460 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:40.410410 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:44.191551 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:44.191527 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q"] Jun 09 05:38:44.211300 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:38:44.211276 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf428f7c2_d4e8_422a_b08b_b883dccfb7f1.slice/crio-80ec84b467e354a9e05fc7f8356590b9d268b93072b10ec32ff85c57fb332033 WatchSource:0}: Error finding container 80ec84b467e354a9e05fc7f8356590b9d268b93072b10ec32ff85c57fb332033: Status 404 returned error can't find the container with id 80ec84b467e354a9e05fc7f8356590b9d268b93072b10ec32ff85c57fb332033 Jun 09 05:38:44.241999 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:44.241977 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:44.244245 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:44.244215 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:44.325897 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:44.325827 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbq28\" (UniqueName: \"kubernetes.io/projected/6517576c-1c63-44d0-ad4b-d7a7e6c02fce-kube-api-access-hbq28\") pod \"6517576c-1c63-44d0-ad4b-d7a7e6c02fce\" (UID: \"6517576c-1c63-44d0-ad4b-d7a7e6c02fce\") " Jun 09 05:38:44.327808 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:44.327784 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6517576c-1c63-44d0-ad4b-d7a7e6c02fce-kube-api-access-hbq28" (OuterVolumeSpecName: "kube-api-access-hbq28") pod "6517576c-1c63-44d0-ad4b-d7a7e6c02fce" (UID: "6517576c-1c63-44d0-ad4b-d7a7e6c02fce"). InnerVolumeSpecName "kube-api-access-hbq28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:38:44.426381 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:44.426349 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbq28\" (UniqueName: \"kubernetes.io/projected/6517576c-1c63-44d0-ad4b-d7a7e6c02fce-kube-api-access-hbq28\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:38:44.799006 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:44.798976 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" path="/var/lib/kubelet/pods/6517576c-1c63-44d0-ad4b-d7a7e6c02fce/volumes" Jun 09 05:38:45.049176 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.049102 2567 generic.go:356] "Generic (PLEG): container finished" podID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" containerID="5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b" exitCode=0 Jun 09 05:38:45.049176 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.049146 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:45.049412 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.049212 2567 scope.go:117] "RemoveContainer" containerID="5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b" Jun 09 05:38:45.050935 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.050899 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" event={"ID":"826c9a71-f354-4554-bbdb-d561b811b26c","Type":"ContainerStarted","Data":"f58a9ec02112c75cb373f8573a56a549bfafe9b756e28955db386c0a9b06df1e"} Jun 09 05:38:45.051618 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:45.051590 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:45.052551 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.052528 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" event={"ID":"f428f7c2-d4e8-422a-b08b-b883dccfb7f1","Type":"ContainerStarted","Data":"42aaf33c66e2e38012254dc2377da12f579daf12cbcd27a75441e3a7aad5c052"} Jun 09 05:38:45.052670 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.052572 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" event={"ID":"f428f7c2-d4e8-422a-b08b-b883dccfb7f1","Type":"ContainerStarted","Data":"80ec84b467e354a9e05fc7f8356590b9d268b93072b10ec32ff85c57fb332033"} Jun 09 05:38:45.052670 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.052639 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:45.065128 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.064963 2567 scope.go:117] "RemoveContainer" containerID="5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b" Jun 09 05:38:45.065264 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:45.065242 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b\": container with ID starting with 5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b not found: ID does not exist" containerID="5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b" Jun 09 05:38:45.065308 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.065271 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b"} err="failed to get container status \"5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b\": rpc error: code = NotFound desc = could not find container \"5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b\": container with ID starting with 5e0d755c5e283e626723d1af4bb477fc7b1a3f85455ae992f4418d2fb2b5950b not found: ID does not exist" Jun 09 05:38:45.067244 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.067202 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-v6krc" podStartSLOduration=1.7782692 podStartE2EDuration="24.067189722s" totalImagesPullingTime="22.288920522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:38:45.065931175 +0000 UTC m=+598.760274168" watchObservedRunningTime="2026-06-09 05:38:45.067189722 +0000 UTC m=+598.761532700" Jun 09 05:38:45.067653 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:45.067632 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:45.069404 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:45.069365 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:45.090220 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:45.090180 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" podStartSLOduration=14.090167265 podStartE2EDuration="14.090167265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:38:45.089266594 +0000 UTC m=+598.783609573" watchObservedRunningTime="2026-06-09 05:38:45.090167265 +0000 UTC m=+598.784510244" Jun 09 05:38:46.675665 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:46.675633 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:46.766494 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:46.766464 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:38:46.767061 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:46.767039 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:38:46.772275 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:46.772218 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:46.797594 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:38:46.797560 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m9jkd\" is forbidden: User \"system:node:ip-10-0-135-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-121.ec2.internal' and this object" podUID="6517576c-1c63-44d0-ad4b-d7a7e6c02fce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m9jkd" Jun 09 05:38:47.700129 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:47.700096 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:48.214351 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:48.214314 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:49.845255 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:49.845217 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:49.898902 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:49.898874 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:53.234864 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:53.234810 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:54.321068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:54.321034 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:56.060273 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:56.060240 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d2s2q" Jun 09 05:38:56.246412 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:56.246357 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:38:56.353806 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:38:56.353726 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:00.180439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:00.178986 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:00.235604 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:00.235572 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:00.335718 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:00.335687 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:03.741106 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:03.741073 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:04.631693 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:04.631660 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:04.726222 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:04.726188 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:04.780150 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:04.780114 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:05.474872 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.474836 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:05.575351 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.575316 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:05.655795 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.655763 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-q679k"] Jun 09 05:39:05.695427 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.695366 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-q679k"] Jun 09 05:39:05.695580 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.695533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:39:05.697926 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.697908 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9dpmn\"" Jun 09 05:39:05.702580 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.702555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx4df\" (UniqueName: \"kubernetes.io/projected/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3-kube-api-access-dx4df\") pod \"authorino-75d556bd77-q679k\" (UID: \"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3\") " pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:39:05.803567 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.803467 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx4df\" (UniqueName: \"kubernetes.io/projected/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3-kube-api-access-dx4df\") pod \"authorino-75d556bd77-q679k\" (UID: \"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3\") " pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:39:05.812312 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:05.812285 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx4df\" (UniqueName: \"kubernetes.io/projected/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3-kube-api-access-dx4df\") pod \"authorino-75d556bd77-q679k\" (UID: \"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3\") " pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:39:06.004349 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:06.004316 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:39:06.131588 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:06.131551 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-q679k"] Jun 09 05:39:06.134432 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:39:06.134401 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd88fb17e_1a23_4eb1_99a7_7b8d571a7db3.slice/crio-c87f09be205ee1b28e1ad3d2d9808aaa5dc88ab052a1bbb5c5e98c6e19fac95c WatchSource:0}: Error finding container c87f09be205ee1b28e1ad3d2d9808aaa5dc88ab052a1bbb5c5e98c6e19fac95c: Status 404 returned error can't find the container with id c87f09be205ee1b28e1ad3d2d9808aaa5dc88ab052a1bbb5c5e98c6e19fac95c Jun 09 05:39:06.148964 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:06.148937 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-q679k" event={"ID":"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3","Type":"ContainerStarted","Data":"c87f09be205ee1b28e1ad3d2d9808aaa5dc88ab052a1bbb5c5e98c6e19fac95c"} Jun 09 05:39:08.234258 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:08.234222 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:08.334339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:08.334298 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:09.163312 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:09.163280 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-q679k" event={"ID":"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3","Type":"ContainerStarted","Data":"a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe"} Jun 09 05:39:09.176506 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:09.176458 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-q679k" podStartSLOduration=1.24034185 podStartE2EDuration="4.176443148s" totalImagesPullingTime="2.936101298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:39:09.174578128 +0000 UTC m=+622.868921107" watchObservedRunningTime="2026-06-09 05:39:09.176443148 +0000 UTC m=+622.870786127" Jun 09 05:39:14.628539 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:14.628504 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:14.681401 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:14.681362 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:15.231512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:15.231482 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:15.331922 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:15.331889 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:18.505124 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:18.505091 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:18.601673 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:18.601632 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:25.747959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:25.747923 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:28.117153 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:28.117114 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:35.577445 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:35.577404 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:35.672939 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:35.672902 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:35.803588 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:35.803555 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:38.438732 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:38.438691 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:38.539853 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:38.539817 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:46.775361 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:46.770520 2567 scope.go:117] "RemoveContainer" containerID="39ee5c93882fe22bb7a41d978cd256d2d052b7c3ad5b68a03be55d0f36be7c21" Jun 09 05:39:58.271705 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.271662 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:58.386118 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.386073 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:39:58.777101 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.777068 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:39:58.782550 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.782533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 05:39:58.785589 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.785251 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 05:39:58.785589 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.785265 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 05:39:58.785589 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.785493 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-sqckg\"" Jun 09 05:39:58.785997 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.785968 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 05:39:58.791531 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.790481 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:39:58.853412 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.853369 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxddn\" (UniqueName: \"kubernetes.io/projected/740dcfac-761e-42d5-9f81-e044ef5fa615-kube-api-access-nxddn\") pod \"maas-keycloak-0\" (UID: \"740dcfac-761e-42d5-9f81-e044ef5fa615\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:39:58.954151 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.954112 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxddn\" (UniqueName: \"kubernetes.io/projected/740dcfac-761e-42d5-9f81-e044ef5fa615-kube-api-access-nxddn\") pod \"maas-keycloak-0\" (UID: \"740dcfac-761e-42d5-9f81-e044ef5fa615\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:39:58.963084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:58.963050 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxddn\" (UniqueName: \"kubernetes.io/projected/740dcfac-761e-42d5-9f81-e044ef5fa615-kube-api-access-nxddn\") pod \"maas-keycloak-0\" (UID: \"740dcfac-761e-42d5-9f81-e044ef5fa615\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:39:59.096215 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:59.096138 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 05:39:59.225726 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:59.225700 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:39:59.227189 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:39:59.227163 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740dcfac_761e_42d5_9f81_e044ef5fa615.slice/crio-19e7eae404719a503e66672f9ca902c5aeca7aaf52efc718225f7c850fca45f5 WatchSource:0}: Error finding container 19e7eae404719a503e66672f9ca902c5aeca7aaf52efc718225f7c850fca45f5: Status 404 returned error can't find the container with id 19e7eae404719a503e66672f9ca902c5aeca7aaf52efc718225f7c850fca45f5 Jun 09 05:39:59.383581 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:39:59.383546 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"740dcfac-761e-42d5-9f81-e044ef5fa615","Type":"ContainerStarted","Data":"19e7eae404719a503e66672f9ca902c5aeca7aaf52efc718225f7c850fca45f5"} Jun 09 05:40:05.415127 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:05.415091 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"740dcfac-761e-42d5-9f81-e044ef5fa615","Type":"ContainerStarted","Data":"2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985"} Jun 09 05:40:05.431853 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:05.431536 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.299002347 podStartE2EDuration="7.431514591s" totalImagesPullingTime="5.132512244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:40:05.430459025 +0000 UTC m=+679.124802008" watchObservedRunningTime="2026-06-09 05:40:05.431514591 +0000 UTC m=+679.125857570" Jun 09 05:40:06.096871 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:06.096820 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 05:40:06.098921 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:06.098873 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:07.096582 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:07.096538 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:08.096767 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:08.096711 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:09.097294 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:09.097243 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 05:40:09.097916 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:09.097719 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:10.096807 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:10.096752 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:11.096950 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:11.096898 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:12.097246 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:12.097187 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:13.096807 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:13.096756 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:14.096959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:14.096908 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:15.097399 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:15.097333 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:16.097315 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:16.097260 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:17.097289 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:17.097240 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.27:9000/health/started\": dial tcp 10.132.0.27:9000: connect: connection refused" Jun 09 05:40:18.213555 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:18.213504 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 05:40:18.227162 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:18.227116 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 05:40:28.218855 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:28.218817 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 05:40:28.856352 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:28.856311 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:29.542258 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:29.542223 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:31.633252 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:31.633216 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-q679k"] Jun 09 05:40:31.633684 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:31.633400 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-q679k" podUID="d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" containerName="authorino" containerID="cri-o://a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe" gracePeriod=30 Jun 09 05:40:31.876264 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:31.876240 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:40:32.059718 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.059633 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx4df\" (UniqueName: \"kubernetes.io/projected/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3-kube-api-access-dx4df\") pod \"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3\" (UID: \"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3\") " Jun 09 05:40:32.061637 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.061609 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3-kube-api-access-dx4df" (OuterVolumeSpecName: "kube-api-access-dx4df") pod "d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" (UID: "d88fb17e-1a23-4eb1-99a7-7b8d571a7db3"). InnerVolumeSpecName "kube-api-access-dx4df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:40:32.161116 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.161084 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx4df\" (UniqueName: \"kubernetes.io/projected/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3-kube-api-access-dx4df\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:40:32.559039 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.559004 2567 generic.go:356] "Generic (PLEG): container finished" podID="d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" containerID="a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe" exitCode=0 Jun 09 05:40:32.559209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.559046 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-q679k" Jun 09 05:40:32.559209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.559047 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-q679k" event={"ID":"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3","Type":"ContainerDied","Data":"a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe"} Jun 09 05:40:32.559209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.559143 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-q679k" event={"ID":"d88fb17e-1a23-4eb1-99a7-7b8d571a7db3","Type":"ContainerDied","Data":"c87f09be205ee1b28e1ad3d2d9808aaa5dc88ab052a1bbb5c5e98c6e19fac95c"} Jun 09 05:40:32.559209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.559163 2567 scope.go:117] "RemoveContainer" containerID="a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe" Jun 09 05:40:32.568090 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.568067 2567 scope.go:117] "RemoveContainer" containerID="a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe" Jun 09 05:40:32.568365 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:40:32.568344 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe\": container with ID starting with a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe not found: ID does not exist" containerID="a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe" Jun 09 05:40:32.568498 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.568373 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe"} err="failed to get container status \"a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe\": rpc error: code = NotFound desc = could not find container \"a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe\": container with ID starting with a4bbcc6fb9c99fbd2aa303cfa2707c084c542bf691edd5dd3e071838ccb3c5fe not found: ID does not exist" Jun 09 05:40:32.578334 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.578309 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-q679k"] Jun 09 05:40:32.584008 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.583986 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-q679k"] Jun 09 05:40:32.799134 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:32.799100 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" path="/var/lib/kubelet/pods/d88fb17e-1a23-4eb1-99a7-7b8d571a7db3/volumes" Jun 09 05:40:35.040211 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.040179 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:35.060817 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.060780 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-ntz5k"] Jun 09 05:40:35.061111 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.061095 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" containerName="authorino" Jun 09 05:40:35.061111 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.061110 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" containerName="authorino" Jun 09 05:40:35.061222 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.061175 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d88fb17e-1a23-4eb1-99a7-7b8d571a7db3" containerName="authorino" Jun 09 05:40:35.064289 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.064272 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.066964 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.066924 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 05:40:35.067304 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.067285 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xfcpb\"" Jun 09 05:40:35.067445 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.067396 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 05:40:35.073795 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.073770 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-ntz5k"] Jun 09 05:40:35.085419 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.085379 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.085545 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.085435 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvk6\" (UniqueName: \"kubernetes.io/projected/094481c1-17ed-4412-8343-0aac8ded6839-kube-api-access-9bvk6\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.139000 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.138961 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:35.186582 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.186548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.186744 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.186587 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvk6\" (UniqueName: \"kubernetes.io/projected/094481c1-17ed-4412-8343-0aac8ded6839-kube-api-access-9bvk6\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.186744 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:40:35.186698 2567 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 05:40:35.186835 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:40:35.186763 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert podName:094481c1-17ed-4412-8343-0aac8ded6839 nodeName:}" failed. No retries permitted until 2026-06-09 05:40:35.68674542 +0000 UTC m=+709.381088378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert") pod "maas-controller-7bbdb77c66-ntz5k" (UID: "094481c1-17ed-4412-8343-0aac8ded6839") : secret "maas-controller-webhook-cert" not found Jun 09 05:40:35.196627 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.196603 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvk6\" (UniqueName: \"kubernetes.io/projected/094481c1-17ed-4412-8343-0aac8ded6839-kube-api-access-9bvk6\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.214494 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.214460 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:35.224321 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.224298 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cbcff6cff-xswxm"] Jun 09 05:40:35.226854 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.226838 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.235930 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.235905 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cbcff6cff-xswxm"] Jun 09 05:40:35.287497 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.287467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8qf\" (UniqueName: \"kubernetes.io/projected/01a9d0e9-e88f-4ec4-b672-705434fe63e1-kube-api-access-ql8qf\") pod \"maas-controller-6cbcff6cff-xswxm\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.287620 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.287542 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01a9d0e9-e88f-4ec4-b672-705434fe63e1-webhook-cert\") pod \"maas-controller-6cbcff6cff-xswxm\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.332317 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.332240 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:35.388500 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.388451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8qf\" (UniqueName: \"kubernetes.io/projected/01a9d0e9-e88f-4ec4-b672-705434fe63e1-kube-api-access-ql8qf\") pod \"maas-controller-6cbcff6cff-xswxm\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.388667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.388539 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01a9d0e9-e88f-4ec4-b672-705434fe63e1-webhook-cert\") pod \"maas-controller-6cbcff6cff-xswxm\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.390866 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.390841 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01a9d0e9-e88f-4ec4-b672-705434fe63e1-webhook-cert\") pod \"maas-controller-6cbcff6cff-xswxm\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.396932 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.396905 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8qf\" (UniqueName: \"kubernetes.io/projected/01a9d0e9-e88f-4ec4-b672-705434fe63e1-kube-api-access-ql8qf\") pod \"maas-controller-6cbcff6cff-xswxm\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.538532 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.538498 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:35.671256 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.671231 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cbcff6cff-xswxm"] Jun 09 05:40:35.673187 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:40:35.673163 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a9d0e9_e88f_4ec4_b672_705434fe63e1.slice/crio-5b2deaa40d51f6d62090dc8f32934c85ca646a47135914d336f02006e2845289 WatchSource:0}: Error finding container 5b2deaa40d51f6d62090dc8f32934c85ca646a47135914d336f02006e2845289: Status 404 returned error can't find the container with id 5b2deaa40d51f6d62090dc8f32934c85ca646a47135914d336f02006e2845289 Jun 09 05:40:35.692093 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.692064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.694328 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.694304 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert\") pod \"maas-controller-7bbdb77c66-ntz5k\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:35.977231 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:35.977200 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:36.105626 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:36.105601 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-ntz5k"] Jun 09 05:40:36.107303 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:40:36.107271 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094481c1_17ed_4412_8343_0aac8ded6839.slice/crio-ed57c21c7d8302fe958c145144af73432bd8506ac51fef04f6a9e052f49631a8 WatchSource:0}: Error finding container ed57c21c7d8302fe958c145144af73432bd8506ac51fef04f6a9e052f49631a8: Status 404 returned error can't find the container with id ed57c21c7d8302fe958c145144af73432bd8506ac51fef04f6a9e052f49631a8 Jun 09 05:40:36.586363 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:36.586287 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" event={"ID":"094481c1-17ed-4412-8343-0aac8ded6839","Type":"ContainerStarted","Data":"ed57c21c7d8302fe958c145144af73432bd8506ac51fef04f6a9e052f49631a8"} Jun 09 05:40:36.588688 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:36.588595 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" event={"ID":"01a9d0e9-e88f-4ec4-b672-705434fe63e1","Type":"ContainerStarted","Data":"5b2deaa40d51f6d62090dc8f32934c85ca646a47135914d336f02006e2845289"} Jun 09 05:40:38.474285 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.474250 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:38.597023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.596938 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:40:38.601036 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.600996 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" event={"ID":"094481c1-17ed-4412-8343-0aac8ded6839","Type":"ContainerStarted","Data":"0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa"} Jun 09 05:40:38.601209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.601075 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:38.602798 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.602761 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" event={"ID":"01a9d0e9-e88f-4ec4-b672-705434fe63e1","Type":"ContainerStarted","Data":"b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20"} Jun 09 05:40:38.602909 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.602841 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:38.634239 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.634186 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" podStartSLOduration=1.455169197 podStartE2EDuration="3.634170994s" totalImagesPullingTime="2.179001797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:40:38.631684638 +0000 UTC m=+712.326027618" watchObservedRunningTime="2026-06-09 05:40:38.634170994 +0000 UTC m=+712.328513973" Jun 09 05:40:38.664040 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:38.663992 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" podStartSLOduration=1.060711949 podStartE2EDuration="3.663976942s" totalImagesPullingTime="2.603264993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:40:38.663968777 +0000 UTC m=+712.358311757" watchObservedRunningTime="2026-06-09 05:40:38.663976942 +0000 UTC m=+712.358319921" Jun 09 05:40:49.612474 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:49.612446 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:49.612862 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:49.612513 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:40:49.684126 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:49.684094 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-ntz5k"] Jun 09 05:40:49.684337 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:49.684315 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" podUID="094481c1-17ed-4412-8343-0aac8ded6839" containerName="manager" containerID="cri-o://0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa" gracePeriod=30 Jun 09 05:40:49.924658 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:49.924631 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:50.019475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.019422 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bvk6\" (UniqueName: \"kubernetes.io/projected/094481c1-17ed-4412-8343-0aac8ded6839-kube-api-access-9bvk6\") pod \"094481c1-17ed-4412-8343-0aac8ded6839\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " Jun 09 05:40:50.019677 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.019519 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert\") pod \"094481c1-17ed-4412-8343-0aac8ded6839\" (UID: \"094481c1-17ed-4412-8343-0aac8ded6839\") " Jun 09 05:40:50.021629 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.021597 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "094481c1-17ed-4412-8343-0aac8ded6839" (UID: "094481c1-17ed-4412-8343-0aac8ded6839"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:40:50.021765 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.021714 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/094481c1-17ed-4412-8343-0aac8ded6839-kube-api-access-9bvk6" (OuterVolumeSpecName: "kube-api-access-9bvk6") pod "094481c1-17ed-4412-8343-0aac8ded6839" (UID: "094481c1-17ed-4412-8343-0aac8ded6839"). InnerVolumeSpecName "kube-api-access-9bvk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:40:50.120377 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.120333 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bvk6\" (UniqueName: \"kubernetes.io/projected/094481c1-17ed-4412-8343-0aac8ded6839-kube-api-access-9bvk6\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:40:50.120377 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.120374 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/094481c1-17ed-4412-8343-0aac8ded6839-webhook-cert\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:40:50.664964 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.664925 2567 generic.go:356] "Generic (PLEG): container finished" podID="094481c1-17ed-4412-8343-0aac8ded6839" containerID="0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa" exitCode=0 Jun 09 05:40:50.664964 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.664967 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" Jun 09 05:40:50.665467 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.665014 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" event={"ID":"094481c1-17ed-4412-8343-0aac8ded6839","Type":"ContainerDied","Data":"0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa"} Jun 09 05:40:50.665467 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.665059 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdb77c66-ntz5k" event={"ID":"094481c1-17ed-4412-8343-0aac8ded6839","Type":"ContainerDied","Data":"ed57c21c7d8302fe958c145144af73432bd8506ac51fef04f6a9e052f49631a8"} Jun 09 05:40:50.665467 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.665082 2567 scope.go:117] "RemoveContainer" containerID="0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa" Jun 09 05:40:50.673627 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.673610 2567 scope.go:117] "RemoveContainer" containerID="0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa" Jun 09 05:40:50.673858 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:40:50.673841 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa\": container with ID starting with 0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa not found: ID does not exist" containerID="0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa" Jun 09 05:40:50.673906 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.673869 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa"} err="failed to get container status \"0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa\": rpc error: code = NotFound desc = could not find container \"0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa\": container with ID starting with 0d41e4f9da4cdcd35cda4f8cb24685af80a069e8c0c6438ea7af663a2c5759fa not found: ID does not exist" Jun 09 05:40:50.684956 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.684935 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-ntz5k"] Jun 09 05:40:50.688452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.688432 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bbdb77c66-ntz5k"] Jun 09 05:40:50.798851 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:40:50.798813 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="094481c1-17ed-4412-8343-0aac8ded6839" path="/var/lib/kubelet/pods/094481c1-17ed-4412-8343-0aac8ded6839/volumes" Jun 09 05:41:09.386790 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.386758 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:41:09.441701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.441669 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-76cdcb8cdb-6dvt9"] Jun 09 05:41:09.442016 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.442003 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="094481c1-17ed-4412-8343-0aac8ded6839" containerName="manager" Jun 09 05:41:09.442063 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.442020 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="094481c1-17ed-4412-8343-0aac8ded6839" containerName="manager" Jun 09 05:41:09.442118 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.442108 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="094481c1-17ed-4412-8343-0aac8ded6839" containerName="manager" Jun 09 05:41:09.444225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.444209 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.446730 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.446706 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 09 05:41:09.446730 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.446725 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w8scx\"" Jun 09 05:41:09.454825 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.454800 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-76cdcb8cdb-6dvt9"] Jun 09 05:41:09.586435 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.586368 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pdhx\" (UniqueName: \"kubernetes.io/projected/f7f81a0b-8dc9-446f-8445-7216ba7df95b-kube-api-access-9pdhx\") pod \"maas-api-76cdcb8cdb-6dvt9\" (UID: \"f7f81a0b-8dc9-446f-8445-7216ba7df95b\") " pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.586644 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.586519 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7f81a0b-8dc9-446f-8445-7216ba7df95b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-6dvt9\" (UID: \"f7f81a0b-8dc9-446f-8445-7216ba7df95b\") " pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.588246 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.588215 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:41:09.687043 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.686961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7f81a0b-8dc9-446f-8445-7216ba7df95b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-6dvt9\" (UID: \"f7f81a0b-8dc9-446f-8445-7216ba7df95b\") " pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.687043 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.687021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pdhx\" (UniqueName: \"kubernetes.io/projected/f7f81a0b-8dc9-446f-8445-7216ba7df95b-kube-api-access-9pdhx\") pod \"maas-api-76cdcb8cdb-6dvt9\" (UID: \"f7f81a0b-8dc9-446f-8445-7216ba7df95b\") " pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.689606 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.689581 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7f81a0b-8dc9-446f-8445-7216ba7df95b-maas-api-tls\") pod \"maas-api-76cdcb8cdb-6dvt9\" (UID: \"f7f81a0b-8dc9-446f-8445-7216ba7df95b\") " pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.694451 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.694418 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pdhx\" (UniqueName: \"kubernetes.io/projected/f7f81a0b-8dc9-446f-8445-7216ba7df95b-kube-api-access-9pdhx\") pod \"maas-api-76cdcb8cdb-6dvt9\" (UID: \"f7f81a0b-8dc9-446f-8445-7216ba7df95b\") " pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.755603 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.755568 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:09.822105 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.822022 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:41:09.888876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.888844 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-76cdcb8cdb-6dvt9"] Jun 09 05:41:09.890919 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:41:09.890893 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f81a0b_8dc9_446f_8445_7216ba7df95b.slice/crio-62468dd19deaf9a74f0518ee660a27ecd433f5204df172b29508aac19e808bfa WatchSource:0}: Error finding container 62468dd19deaf9a74f0518ee660a27ecd433f5204df172b29508aac19e808bfa: Status 404 returned error can't find the container with id 62468dd19deaf9a74f0518ee660a27ecd433f5204df172b29508aac19e808bfa Jun 09 05:41:09.893667 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.893640 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:41:09.930402 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.927763 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-jjbkm"] Jun 09 05:41:09.931844 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.931821 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:09.934275 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.934256 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ft2tz\"" Jun 09 05:41:09.934376 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.934264 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 09 05:41:09.937109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.937050 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-jjbkm"] Jun 09 05:41:09.964581 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.964545 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-mrvh5"] Jun 09 05:41:09.966796 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.966780 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:09.976955 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:09.976932 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-mrvh5"] Jun 09 05:41:10.091124 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.091091 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc59w\" (UniqueName: \"kubernetes.io/projected/4e0439f0-07d6-42db-9d09-d36be3a708e6-kube-api-access-tc59w\") pod \"payload-pre-processing-67d7689b65-jjbkm\" (UID: \"4e0439f0-07d6-42db-9d09-d36be3a708e6\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:10.091291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.091148 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftmf7\" (UniqueName: \"kubernetes.io/projected/c626053c-b8ea-4714-afed-2ab6a9b12901-kube-api-access-ftmf7\") pod \"payload-processing-7c6c859784-mrvh5\" (UID: \"c626053c-b8ea-4714-afed-2ab6a9b12901\") " pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:10.191955 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.191850 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc59w\" (UniqueName: \"kubernetes.io/projected/4e0439f0-07d6-42db-9d09-d36be3a708e6-kube-api-access-tc59w\") pod \"payload-pre-processing-67d7689b65-jjbkm\" (UID: \"4e0439f0-07d6-42db-9d09-d36be3a708e6\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:10.191955 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.191926 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftmf7\" (UniqueName: \"kubernetes.io/projected/c626053c-b8ea-4714-afed-2ab6a9b12901-kube-api-access-ftmf7\") pod \"payload-processing-7c6c859784-mrvh5\" (UID: \"c626053c-b8ea-4714-afed-2ab6a9b12901\") " pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:10.200529 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.200498 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc59w\" (UniqueName: \"kubernetes.io/projected/4e0439f0-07d6-42db-9d09-d36be3a708e6-kube-api-access-tc59w\") pod \"payload-pre-processing-67d7689b65-jjbkm\" (UID: \"4e0439f0-07d6-42db-9d09-d36be3a708e6\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:10.200724 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.200707 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftmf7\" (UniqueName: \"kubernetes.io/projected/c626053c-b8ea-4714-afed-2ab6a9b12901-kube-api-access-ftmf7\") pod \"payload-processing-7c6c859784-mrvh5\" (UID: \"c626053c-b8ea-4714-afed-2ab6a9b12901\") " pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:10.243617 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.243586 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:10.277590 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.277563 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:10.378213 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.378186 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-jjbkm"] Jun 09 05:41:10.380310 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:41:10.380286 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0439f0_07d6_42db_9d09_d36be3a708e6.slice/crio-5e8562047d9d7632506ec083fb13291e5f59966389238aff16c804af907f633e WatchSource:0}: Error finding container 5e8562047d9d7632506ec083fb13291e5f59966389238aff16c804af907f633e: Status 404 returned error can't find the container with id 5e8562047d9d7632506ec083fb13291e5f59966389238aff16c804af907f633e Jun 09 05:41:10.417966 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.417940 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-mrvh5"] Jun 09 05:41:10.421603 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:41:10.421567 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc626053c_b8ea_4714_afed_2ab6a9b12901.slice/crio-c4602e555a8fa19d3d4a483f8d553d39d943a50f5a318458a1a1d32329386d6c WatchSource:0}: Error finding container c4602e555a8fa19d3d4a483f8d553d39d943a50f5a318458a1a1d32329386d6c: Status 404 returned error can't find the container with id c4602e555a8fa19d3d4a483f8d553d39d943a50f5a318458a1a1d32329386d6c Jun 09 05:41:10.753017 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.752977 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" event={"ID":"4e0439f0-07d6-42db-9d09-d36be3a708e6","Type":"ContainerStarted","Data":"5e8562047d9d7632506ec083fb13291e5f59966389238aff16c804af907f633e"} Jun 09 05:41:10.754166 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.754146 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" event={"ID":"f7f81a0b-8dc9-446f-8445-7216ba7df95b","Type":"ContainerStarted","Data":"62468dd19deaf9a74f0518ee660a27ecd433f5204df172b29508aac19e808bfa"} Jun 09 05:41:10.755243 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:10.755221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"c4602e555a8fa19d3d4a483f8d553d39d943a50f5a318458a1a1d32329386d6c"} Jun 09 05:41:12.765775 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.765739 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"b22e5c08cb8e0d0d18b668b39a18ed42faca59d82c37f1dd034e46c41486d54c"} Jun 09 05:41:12.766285 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.765832 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:12.767265 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.767237 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" event={"ID":"4e0439f0-07d6-42db-9d09-d36be3a708e6","Type":"ContainerStarted","Data":"0e6e872ed91386d843f23bc8cd683cadaffb4fb2ed8e8550ac3178fa775fd395"} Jun 09 05:41:12.767412 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.767283 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:12.768626 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.768607 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" event={"ID":"f7f81a0b-8dc9-446f-8445-7216ba7df95b","Type":"ContainerStarted","Data":"e99e0e6c31aa11b224b9f9ac041858df0c45a4c73a0daa9082eed93816418c60"} Jun 09 05:41:12.768737 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.768727 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:12.781648 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.781608 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podStartSLOduration=2.196364432 podStartE2EDuration="3.781596056s" totalImagesPullingTime="1.585231624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:41:12.779913553 +0000 UTC m=+746.474256529" watchObservedRunningTime="2026-06-09 05:41:12.781596056 +0000 UTC m=+746.475939034" Jun 09 05:41:12.794848 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.794801 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" podStartSLOduration=2.171051031 podStartE2EDuration="3.794785068s" totalImagesPullingTime="1.623734037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:41:12.792190542 +0000 UTC m=+746.486533523" watchObservedRunningTime="2026-06-09 05:41:12.794785068 +0000 UTC m=+746.489128048" Jun 09 05:41:12.807763 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:12.807722 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" podStartSLOduration=1.693940187 podStartE2EDuration="3.807708676s" totalImagesPullingTime="2.113768489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:41:12.806555019 +0000 UTC m=+746.500897999" watchObservedRunningTime="2026-06-09 05:41:12.807708676 +0000 UTC m=+746.502051655" Jun 09 05:41:18.779763 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:18.779723 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-76cdcb8cdb-6dvt9" Jun 09 05:41:20.568409 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:20.566112 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:41:20.568409 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:20.566441 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" containerID="cri-o://2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985" gracePeriod=30 Jun 09 05:41:22.612451 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.612427 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:22.697157 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.697127 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxddn\" (UniqueName: \"kubernetes.io/projected/740dcfac-761e-42d5-9f81-e044ef5fa615-kube-api-access-nxddn\") pod \"740dcfac-761e-42d5-9f81-e044ef5fa615\" (UID: \"740dcfac-761e-42d5-9f81-e044ef5fa615\") " Jun 09 05:41:22.699218 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.699184 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740dcfac-761e-42d5-9f81-e044ef5fa615-kube-api-access-nxddn" (OuterVolumeSpecName: "kube-api-access-nxddn") pod "740dcfac-761e-42d5-9f81-e044ef5fa615" (UID: "740dcfac-761e-42d5-9f81-e044ef5fa615"). InnerVolumeSpecName "kube-api-access-nxddn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:41:22.798344 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.798266 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxddn\" (UniqueName: \"kubernetes.io/projected/740dcfac-761e-42d5-9f81-e044ef5fa615-kube-api-access-nxddn\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:41:22.816027 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.816002 2567 generic.go:356] "Generic (PLEG): container finished" podID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerID="2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985" exitCode=143 Jun 09 05:41:22.816137 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.816042 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:22.816137 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.816093 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"740dcfac-761e-42d5-9f81-e044ef5fa615","Type":"ContainerDied","Data":"2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985"} Jun 09 05:41:22.816211 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.816137 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"740dcfac-761e-42d5-9f81-e044ef5fa615","Type":"ContainerDied","Data":"19e7eae404719a503e66672f9ca902c5aeca7aaf52efc718225f7c850fca45f5"} Jun 09 05:41:22.816211 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.816157 2567 scope.go:117] "RemoveContainer" containerID="2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985" Jun 09 05:41:22.825182 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.825162 2567 scope.go:117] "RemoveContainer" containerID="2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985" Jun 09 05:41:22.825484 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:41:22.825462 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985\": container with ID starting with 2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985 not found: ID does not exist" containerID="2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985" Jun 09 05:41:22.825569 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.825493 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985"} err="failed to get container status \"2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985\": rpc error: code = NotFound desc = could not find container \"2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985\": container with ID starting with 2d5822e4e22f73fe8ad9d3ba00d1cbfe75a940488124149922494e9fb1439985 not found: ID does not exist" Jun 09 05:41:22.832259 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.832236 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:41:22.835721 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.835698 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:41:22.855770 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.855745 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:41:22.856084 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.856070 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" Jun 09 05:41:22.856125 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.856086 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" Jun 09 05:41:22.856163 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.856155 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" containerName="keycloak" Jun 09 05:41:22.858556 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.858538 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:22.860940 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.860922 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 05:41:22.861163 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.861147 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-sqckg\"" Jun 09 05:41:22.861202 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.861179 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 05:41:22.861418 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.861202 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 09 05:41:22.861418 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.861149 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 05:41:22.867345 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.867325 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:41:22.898699 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.898671 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkxvx\" (UniqueName: \"kubernetes.io/projected/3888a05a-9637-41e1-ac5c-ed4640941c32-kube-api-access-qkxvx\") pod \"maas-keycloak-0\" (UID: \"3888a05a-9637-41e1-ac5c-ed4640941c32\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:22.898831 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.898813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/3888a05a-9637-41e1-ac5c-ed4640941c32-test-realms\") pod \"maas-keycloak-0\" (UID: \"3888a05a-9637-41e1-ac5c-ed4640941c32\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:22.999599 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.999564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/3888a05a-9637-41e1-ac5c-ed4640941c32-test-realms\") pod \"maas-keycloak-0\" (UID: \"3888a05a-9637-41e1-ac5c-ed4640941c32\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:22.999741 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:22.999629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkxvx\" (UniqueName: \"kubernetes.io/projected/3888a05a-9637-41e1-ac5c-ed4640941c32-kube-api-access-qkxvx\") pod \"maas-keycloak-0\" (UID: \"3888a05a-9637-41e1-ac5c-ed4640941c32\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:23.000274 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.000257 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/3888a05a-9637-41e1-ac5c-ed4640941c32-test-realms\") pod \"maas-keycloak-0\" (UID: \"3888a05a-9637-41e1-ac5c-ed4640941c32\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:23.007638 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.007605 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkxvx\" (UniqueName: \"kubernetes.io/projected/3888a05a-9637-41e1-ac5c-ed4640941c32-kube-api-access-qkxvx\") pod \"maas-keycloak-0\" (UID: \"3888a05a-9637-41e1-ac5c-ed4640941c32\") " pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:23.168921 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.168890 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:23.296981 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.296957 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 05:41:23.299355 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:41:23.299325 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3888a05a_9637_41e1_ac5c_ed4640941c32.slice/crio-6c22a413abff85b64d552e4570c344644a2190290e386a0f7e03704e66d75177 WatchSource:0}: Error finding container 6c22a413abff85b64d552e4570c344644a2190290e386a0f7e03704e66d75177: Status 404 returned error can't find the container with id 6c22a413abff85b64d552e4570c344644a2190290e386a0f7e03704e66d75177 Jun 09 05:41:23.300814 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.300793 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 05:41:23.776797 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.776551 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:41:23.777241 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.776813 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-jjbkm" Jun 09 05:41:23.825339 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.825298 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"3888a05a-9637-41e1-ac5c-ed4640941c32","Type":"ContainerStarted","Data":"440edf71364f58a3dfe147167e0ca2e9ba291bd909a6ca5a39dedca9177ddaba"} Jun 09 05:41:23.825549 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.825349 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"3888a05a-9637-41e1-ac5c-ed4640941c32","Type":"ContainerStarted","Data":"6c22a413abff85b64d552e4570c344644a2190290e386a0f7e03704e66d75177"} Jun 09 05:41:23.842297 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:23.842246 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.5230826469999998 podStartE2EDuration="1.84223047s" totalImagesPullingTime="319.147823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:41:23.84042632 +0000 UTC m=+757.534769300" watchObservedRunningTime="2026-06-09 05:41:23.84223047 +0000 UTC m=+757.536573449" Jun 09 05:41:24.169894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:24.169856 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:24.171502 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:24.171466 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:24.801581 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:24.801537 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="740dcfac-761e-42d5-9f81-e044ef5fa615" path="/var/lib/kubelet/pods/740dcfac-761e-42d5-9f81-e044ef5fa615/volumes" Jun 09 05:41:25.170209 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:25.170158 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:26.170327 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:26.169904 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:27.169731 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:27.169672 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:28.169424 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:28.169344 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:29.169931 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:29.169885 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:30.170743 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:30.170692 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:31.170207 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:31.170142 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:32.169629 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:32.169573 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:33.169019 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:33.168972 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:33.169429 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:33.169360 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:34.170125 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:34.170071 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:35.169824 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:35.169782 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:36.170065 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:36.170000 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.33:9000/health/started\": dial tcp 10.132.0.33:9000: connect: connection refused" Jun 09 05:41:37.290056 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:37.290028 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:37.305861 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:37.305816 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="3888a05a-9637-41e1-ac5c-ed4640941c32" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 05:41:47.296801 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:47.296765 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 05:41:59.541496 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:59.541458 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:41:59.642591 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:41:59.642555 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:42:03.922751 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:03.922714 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:42:03.973307 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:03.973265 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:42:16.505445 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.505335 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj"] Jun 09 05:42:16.508003 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.507984 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.510783 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.510759 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 05:42:16.511703 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.511683 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 09 05:42:16.511782 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.511687 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fl8ct\"" Jun 09 05:42:16.511959 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.511921 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 05:42:16.519622 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.519599 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj"] Jun 09 05:42:16.531275 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.531252 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:42:16.590770 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.590738 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1dbcd90-9733-44e2-a67c-3a8c950348c0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.590914 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.590785 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.590983 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.590908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.591032 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.591014 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9ljv\" (UniqueName: \"kubernetes.io/projected/c1dbcd90-9733-44e2-a67c-3a8c950348c0-kube-api-access-g9ljv\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.591075 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.591043 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.591121 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.591072 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.691725 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.691692 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.691879 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.691733 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1dbcd90-9733-44e2-a67c-3a8c950348c0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.691879 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.691753 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.691879 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.691793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.691879 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.691831 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9ljv\" (UniqueName: \"kubernetes.io/projected/c1dbcd90-9733-44e2-a67c-3a8c950348c0-kube-api-access-g9ljv\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.691879 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.691849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.692255 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.692221 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.692368 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.692265 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.692368 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.692274 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.693854 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.693834 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1dbcd90-9733-44e2-a67c-3a8c950348c0-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.694153 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.694136 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1dbcd90-9733-44e2-a67c-3a8c950348c0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.699678 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.699657 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9ljv\" (UniqueName: \"kubernetes.io/projected/c1dbcd90-9733-44e2-a67c-3a8c950348c0-kube-api-access-g9ljv\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj\" (UID: \"c1dbcd90-9733-44e2-a67c-3a8c950348c0\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.818418 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.818307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:16.954423 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:16.954396 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj"] Jun 09 05:42:16.956379 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:42:16.956355 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1dbcd90_9733_44e2_a67c_3a8c950348c0.slice/crio-8ef18b40723264755077e86bdbe57edbbf38b8842dd088739896307f91b7d85c WatchSource:0}: Error finding container 8ef18b40723264755077e86bdbe57edbbf38b8842dd088739896307f91b7d85c: Status 404 returned error can't find the container with id 8ef18b40723264755077e86bdbe57edbbf38b8842dd088739896307f91b7d85c Jun 09 05:42:17.099623 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:17.099542 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" event={"ID":"c1dbcd90-9733-44e2-a67c-3a8c950348c0","Type":"ContainerStarted","Data":"8ef18b40723264755077e86bdbe57edbbf38b8842dd088739896307f91b7d85c"} Jun 09 05:42:21.121217 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:21.121183 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" event={"ID":"c1dbcd90-9733-44e2-a67c-3a8c950348c0","Type":"ContainerStarted","Data":"30998373534e4917d79c574dbdcc1907c43af591ea804bab6835ff0ab96bb8f9"} Jun 09 05:42:22.127433 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:22.127382 2567 generic.go:356] "Generic (PLEG): container finished" podID="c1dbcd90-9733-44e2-a67c-3a8c950348c0" containerID="30998373534e4917d79c574dbdcc1907c43af591ea804bab6835ff0ab96bb8f9" exitCode=0 Jun 09 05:42:22.127899 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:22.127465 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" event={"ID":"c1dbcd90-9733-44e2-a67c-3a8c950348c0","Type":"ContainerDied","Data":"30998373534e4917d79c574dbdcc1907c43af591ea804bab6835ff0ab96bb8f9"} Jun 09 05:42:24.139827 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:24.139788 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" event={"ID":"c1dbcd90-9733-44e2-a67c-3a8c950348c0","Type":"ContainerStarted","Data":"7777a0399b71aff763510741cc377aa57ab816a9de896e91bb55f90b41b0a5c6"} Jun 09 05:42:24.158367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:24.158324 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" podStartSLOduration=2.312145193 podStartE2EDuration="8.158310086s" totalImagesPullingTime="4.632114552s" totalInitContainerRuntime="1.214050341s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:42:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:42:24.156685668 +0000 UTC m=+817.851028670" watchObservedRunningTime="2026-06-09 05:42:24.158310086 +0000 UTC m=+817.852653064" Jun 09 05:42:26.819402 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:26.819367 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:26.819804 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:26.819433 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:26.832376 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:26.832352 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:27.164661 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:27.164627 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj" Jun 09 05:42:59.540320 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:59.540290 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:42:59.641689 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:42:59.641654 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:03.000343 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.000309 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb"] Jun 09 05:43:03.004023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.004004 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.006491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.006465 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 09 05:43:03.012203 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.012184 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb"] Jun 09 05:43:03.085913 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.085875 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:03.099999 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.099973 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.100074 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.100012 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.100074 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.100038 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.100167 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.100150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd80f705-65ed-4264-8e56-2d2824c78c20-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.100204 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.100184 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rc56\" (UniqueName: \"kubernetes.io/projected/cd80f705-65ed-4264-8e56-2d2824c78c20-kube-api-access-8rc56\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.100242 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.100214 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201178 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd80f705-65ed-4264-8e56-2d2824c78c20-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201178 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201188 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rc56\" (UniqueName: \"kubernetes.io/projected/cd80f705-65ed-4264-8e56-2d2824c78c20-kube-api-access-8rc56\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201418 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201477 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201520 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201577 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201681 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201660 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201791 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201770 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.201933 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.201909 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.203666 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.203643 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd80f705-65ed-4264-8e56-2d2824c78c20-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.203882 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.203863 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd80f705-65ed-4264-8e56-2d2824c78c20-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.208893 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.208867 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rc56\" (UniqueName: \"kubernetes.io/projected/cd80f705-65ed-4264-8e56-2d2824c78c20-kube-api-access-8rc56\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-sd6tb\" (UID: \"cd80f705-65ed-4264-8e56-2d2824c78c20\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.322705 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.322612 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:03.463855 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:03.463826 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb"] Jun 09 05:43:03.465836 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:43:03.465809 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd80f705_65ed_4264_8e56_2d2824c78c20.slice/crio-7011dd94a162edefebf47a9b557e0110f026f71589264487af389c5caf0200fb WatchSource:0}: Error finding container 7011dd94a162edefebf47a9b557e0110f026f71589264487af389c5caf0200fb: Status 404 returned error can't find the container with id 7011dd94a162edefebf47a9b557e0110f026f71589264487af389c5caf0200fb Jun 09 05:43:04.331240 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:04.331200 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" event={"ID":"cd80f705-65ed-4264-8e56-2d2824c78c20","Type":"ContainerStarted","Data":"e302f6e28c244462c54953f3c997c825e28ed2a162bfdf2230bb320212dbd74c"} Jun 09 05:43:04.331711 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:04.331255 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" event={"ID":"cd80f705-65ed-4264-8e56-2d2824c78c20","Type":"ContainerStarted","Data":"7011dd94a162edefebf47a9b557e0110f026f71589264487af389c5caf0200fb"} Jun 09 05:43:09.885193 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.885159 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk"] Jun 09 05:43:09.889140 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.889121 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:09.891439 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.891418 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 09 05:43:09.899624 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.899604 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk"] Jun 09 05:43:09.916666 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.916637 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:09.959351 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.959320 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:09.959501 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.959366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w7ff\" (UniqueName: \"kubernetes.io/projected/adc808e0-58a1-464c-94a8-ece0b85438fa-kube-api-access-2w7ff\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:09.959501 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.959459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adc808e0-58a1-464c-94a8-ece0b85438fa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:09.959597 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.959540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:09.959597 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.959578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:09.959685 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:09.959602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.060594 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.060560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.060776 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.060605 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w7ff\" (UniqueName: \"kubernetes.io/projected/adc808e0-58a1-464c-94a8-ece0b85438fa-kube-api-access-2w7ff\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.060776 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.060728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adc808e0-58a1-464c-94a8-ece0b85438fa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.060902 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.060782 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.060902 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.060820 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.060902 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.060850 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.061279 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.061244 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.061424 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.061285 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.061513 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.061438 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.062956 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.062924 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adc808e0-58a1-464c-94a8-ece0b85438fa-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.063210 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.063189 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adc808e0-58a1-464c-94a8-ece0b85438fa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.067900 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.067875 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w7ff\" (UniqueName: \"kubernetes.io/projected/adc808e0-58a1-464c-94a8-ece0b85438fa-kube-api-access-2w7ff\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk\" (UID: \"adc808e0-58a1-464c-94a8-ece0b85438fa\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.200610 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.200525 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:10.329259 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.329232 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk"] Jun 09 05:43:10.331513 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:43:10.331486 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc808e0_58a1_464c_94a8_ece0b85438fa.slice/crio-6cd3fa6dd391ac319cdec970a384d468a6220c1cb273959d6fa1f69488406f24 WatchSource:0}: Error finding container 6cd3fa6dd391ac319cdec970a384d468a6220c1cb273959d6fa1f69488406f24: Status 404 returned error can't find the container with id 6cd3fa6dd391ac319cdec970a384d468a6220c1cb273959d6fa1f69488406f24 Jun 09 05:43:10.360444 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:10.360405 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" event={"ID":"adc808e0-58a1-464c-94a8-ece0b85438fa","Type":"ContainerStarted","Data":"6cd3fa6dd391ac319cdec970a384d468a6220c1cb273959d6fa1f69488406f24"} Jun 09 05:43:11.367038 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:11.367005 2567 generic.go:356] "Generic (PLEG): container finished" podID="cd80f705-65ed-4264-8e56-2d2824c78c20" containerID="e302f6e28c244462c54953f3c997c825e28ed2a162bfdf2230bb320212dbd74c" exitCode=0 Jun 09 05:43:11.367469 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:11.367077 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" event={"ID":"cd80f705-65ed-4264-8e56-2d2824c78c20","Type":"ContainerDied","Data":"e302f6e28c244462c54953f3c997c825e28ed2a162bfdf2230bb320212dbd74c"} Jun 09 05:43:11.368661 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:11.368640 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" event={"ID":"adc808e0-58a1-464c-94a8-ece0b85438fa","Type":"ContainerStarted","Data":"44399a635bbc6a72265eb7cad288e5929f4b16f4c97ceb69e88c3e7ed9d83820"} Jun 09 05:43:12.376031 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.376000 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" event={"ID":"cd80f705-65ed-4264-8e56-2d2824c78c20","Type":"ContainerStarted","Data":"a0240d28c4e50584c0241c95b624003053b9d2daa211a957654c4fd3b5e2b3af"} Jun 09 05:43:12.377589 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.377565 2567 generic.go:356] "Generic (PLEG): container finished" podID="adc808e0-58a1-464c-94a8-ece0b85438fa" containerID="44399a635bbc6a72265eb7cad288e5929f4b16f4c97ceb69e88c3e7ed9d83820" exitCode=0 Jun 09 05:43:12.377705 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.377652 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" event={"ID":"adc808e0-58a1-464c-94a8-ece0b85438fa","Type":"ContainerDied","Data":"44399a635bbc6a72265eb7cad288e5929f4b16f4c97ceb69e88c3e7ed9d83820"} Jun 09 05:43:12.379396 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.379366 2567 generic.go:356] "Generic (PLEG): container finished" podID="c626053c-b8ea-4714-afed-2ab6a9b12901" containerID="b22e5c08cb8e0d0d18b668b39a18ed42faca59d82c37f1dd034e46c41486d54c" exitCode=1 Jun 09 05:43:12.379474 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.379446 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerDied","Data":"b22e5c08cb8e0d0d18b668b39a18ed42faca59d82c37f1dd034e46c41486d54c"} Jun 09 05:43:12.379711 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.379698 2567 scope.go:117] "RemoveContainer" containerID="b22e5c08cb8e0d0d18b668b39a18ed42faca59d82c37f1dd034e46c41486d54c" Jun 09 05:43:12.394438 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:12.394367 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" podStartSLOduration=2.5672555900000003 podStartE2EDuration="10.394347845s" totalImagesPullingTime="191.83673ms" totalInitContainerRuntime="7.635255525s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:43:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:43:12.392850306 +0000 UTC m=+866.087193285" watchObservedRunningTime="2026-06-09 05:43:12.394347845 +0000 UTC m=+866.088690826" Jun 09 05:43:13.323531 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.323489 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:13.323718 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.323570 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:13.336089 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.336060 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:13.385982 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.385950 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" event={"ID":"adc808e0-58a1-464c-94a8-ece0b85438fa","Type":"ContainerStarted","Data":"8a5f3348cf6fc3a9feab3274eb4f4c0f5668a5e9cd657899bc4598547dfac749"} Jun 09 05:43:13.387752 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.387727 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65"} Jun 09 05:43:13.388070 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.388050 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:43:13.399207 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.399188 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-sd6tb" Jun 09 05:43:13.404025 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:13.403985 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" podStartSLOduration=2.93973154 podStartE2EDuration="4.403972379s" totalImagesPullingTime="186.545546ms" totalInitContainerRuntime="1.277695293s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:43:13.401776794 +0000 UTC m=+867.096119773" watchObservedRunningTime="2026-06-09 05:43:13.403972379 +0000 UTC m=+867.098315357" Jun 09 05:43:14.691338 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.691303 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5"] Jun 09 05:43:14.700144 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.700120 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.703353 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.703331 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 09 05:43:14.705682 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.705654 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5"] Jun 09 05:43:14.710263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.710234 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:14.804578 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.804546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.804578 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.804581 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.804783 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.804614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zq9z\" (UniqueName: \"kubernetes.io/projected/7f2b84cf-96be-4d6f-a701-af28374f22b6-kube-api-access-8zq9z\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.804783 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.804738 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f2b84cf-96be-4d6f-a701-af28374f22b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.804783 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.804768 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.804899 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.804801 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.905617 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.905582 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f2b84cf-96be-4d6f-a701-af28374f22b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.905617 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.905617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.905865 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.905649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.905865 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.905702 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.905865 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.905725 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.905865 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.905757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zq9z\" (UniqueName: \"kubernetes.io/projected/7f2b84cf-96be-4d6f-a701-af28374f22b6-kube-api-access-8zq9z\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.906096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.906043 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.906096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.906077 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.906198 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.906123 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.908109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.908083 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f2b84cf-96be-4d6f-a701-af28374f22b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.908109 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.908102 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f2b84cf-96be-4d6f-a701-af28374f22b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:14.913110 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:14.913086 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zq9z\" (UniqueName: \"kubernetes.io/projected/7f2b84cf-96be-4d6f-a701-af28374f22b6-kube-api-access-8zq9z\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5\" (UID: \"7f2b84cf-96be-4d6f-a701-af28374f22b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:15.012510 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:15.012432 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:43:15.149303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:15.149271 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5"] Jun 09 05:43:15.150374 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:43:15.150348 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2b84cf_96be_4d6f_a701_af28374f22b6.slice/crio-83218c52a10439e39a82d43eccbdfd475196480c603901accf499c12894b2b6e WatchSource:0}: Error finding container 83218c52a10439e39a82d43eccbdfd475196480c603901accf499c12894b2b6e: Status 404 returned error can't find the container with id 83218c52a10439e39a82d43eccbdfd475196480c603901accf499c12894b2b6e Jun 09 05:43:15.400010 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:15.399972 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" event={"ID":"7f2b84cf-96be-4d6f-a701-af28374f22b6","Type":"ContainerStarted","Data":"8559ff2e7f38aeb8fc1f67f8a03b62651c595221fb8cb0927d5680ed60997cfa"} Jun 09 05:43:15.400302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:15.400037 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" event={"ID":"7f2b84cf-96be-4d6f-a701-af28374f22b6","Type":"ContainerStarted","Data":"83218c52a10439e39a82d43eccbdfd475196480c603901accf499c12894b2b6e"} Jun 09 05:43:18.366281 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.366240 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l"] Jun 09 05:43:18.377061 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.377000 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.379452 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.379427 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l"] Jun 09 05:43:18.379648 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.379628 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 09 05:43:18.441351 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.441317 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.441512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.441375 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.441512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.441428 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.441512 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.441492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.441681 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.441531 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2lj\" (UniqueName: \"kubernetes.io/projected/c50027c5-3f58-4001-bfcf-2a50462860f6-kube-api-access-lx2lj\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.441681 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.441558 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50027c5-3f58-4001-bfcf-2a50462860f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.485283 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.485254 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:18.542694 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.542666 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.542876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.542728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.542876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.542751 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.542876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.542800 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.542876 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.542847 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2lj\" (UniqueName: \"kubernetes.io/projected/c50027c5-3f58-4001-bfcf-2a50462860f6-kube-api-access-lx2lj\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.543137 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.542875 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50027c5-3f58-4001-bfcf-2a50462860f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.543206 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.543148 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.543206 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.543179 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.543314 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.543246 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.545096 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.545031 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c50027c5-3f58-4001-bfcf-2a50462860f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.545346 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.545328 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c50027c5-3f58-4001-bfcf-2a50462860f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.553201 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.553179 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2lj\" (UniqueName: \"kubernetes.io/projected/c50027c5-3f58-4001-bfcf-2a50462860f6-kube-api-access-lx2lj\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l\" (UID: \"c50027c5-3f58-4001-bfcf-2a50462860f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.688745 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.688715 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:18.820894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:18.820860 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l"] Jun 09 05:43:18.826949 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:43:18.826919 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50027c5_3f58_4001_bfcf_2a50462860f6.slice/crio-71d37d39fbd63a8cade5eeb470f045db6aad241ffcee8f1f394a0fb7eaa0a7ec WatchSource:0}: Error finding container 71d37d39fbd63a8cade5eeb470f045db6aad241ffcee8f1f394a0fb7eaa0a7ec: Status 404 returned error can't find the container with id 71d37d39fbd63a8cade5eeb470f045db6aad241ffcee8f1f394a0fb7eaa0a7ec Jun 09 05:43:19.422482 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:19.422439 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" event={"ID":"c50027c5-3f58-4001-bfcf-2a50462860f6","Type":"ContainerStarted","Data":"42bd57bd0d14ae170cffa2ccb1b918425a4b8f765b616b168f098d8b50f40493"} Jun 09 05:43:19.422914 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:19.422499 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" event={"ID":"c50027c5-3f58-4001-bfcf-2a50462860f6","Type":"ContainerStarted","Data":"71d37d39fbd63a8cade5eeb470f045db6aad241ffcee8f1f394a0fb7eaa0a7ec"} Jun 09 05:43:20.201598 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:20.201563 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:20.201753 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:20.201627 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:20.214283 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:20.214256 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:20.428832 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:20.428751 2567 generic.go:356] "Generic (PLEG): container finished" podID="c50027c5-3f58-4001-bfcf-2a50462860f6" containerID="42bd57bd0d14ae170cffa2ccb1b918425a4b8f765b616b168f098d8b50f40493" exitCode=0 Jun 09 05:43:20.429233 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:20.428840 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" event={"ID":"c50027c5-3f58-4001-bfcf-2a50462860f6","Type":"ContainerDied","Data":"42bd57bd0d14ae170cffa2ccb1b918425a4b8f765b616b168f098d8b50f40493"} Jun 09 05:43:20.440034 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:20.440012 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk" Jun 09 05:43:21.436420 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:21.436363 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" event={"ID":"c50027c5-3f58-4001-bfcf-2a50462860f6","Type":"ContainerStarted","Data":"efe6a23e18469e9c90b5846762758bdb588eece178126dad0fc7570e80b2d11a"} Jun 09 05:43:21.454737 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:21.454690 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" podStartSLOduration=1.994273218 podStartE2EDuration="3.454675261s" totalImagesPullingTime="170.551154ms" totalInitContainerRuntime="1.289850889s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:43:21.452960436 +0000 UTC m=+875.147303415" watchObservedRunningTime="2026-06-09 05:43:21.454675261 +0000 UTC m=+875.149018239" Jun 09 05:43:21.983009 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:21.982976 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:23.218853 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:23.218811 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:24.394631 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:24.394578 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:43:28.689106 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:28.689076 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:28.689605 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:28.689131 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:28.708453 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:28.702764 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:29.488483 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:29.488455 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l" Jun 09 05:43:46.765704 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:46.765665 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:43:46.798415 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:46.798368 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:43:46.802429 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:46.802396 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:43:46.821568 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:43:46.821520 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:44:19.711764 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:19.711726 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:44:20.845248 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:20.845209 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:44:55.159442 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:55.159406 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:44:58.896206 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:58.896177 2567 generic.go:356] "Generic (PLEG): container finished" podID="7f2b84cf-96be-4d6f-a701-af28374f22b6" containerID="8559ff2e7f38aeb8fc1f67f8a03b62651c595221fb8cb0927d5680ed60997cfa" exitCode=0 Jun 09 05:44:58.896624 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:58.896248 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" event={"ID":"7f2b84cf-96be-4d6f-a701-af28374f22b6","Type":"ContainerDied","Data":"8559ff2e7f38aeb8fc1f67f8a03b62651c595221fb8cb0927d5680ed60997cfa"} Jun 09 05:44:59.903111 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:59.903075 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" event={"ID":"7f2b84cf-96be-4d6f-a701-af28374f22b6","Type":"ContainerStarted","Data":"6aeec86e2f4f2211cfbd7c7d886599fd3008b6a4cebdcfeeaaead65dfb8fc754"} Jun 09 05:44:59.920929 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:44:59.920882 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" podStartSLOduration=2.300293352 podStartE2EDuration="1m45.920868135s" totalImagesPullingTime="177.792475ms" totalInitContainerRuntime="1m43.442782308s" isStatefulPod=false podCreationTimestamp="2026-06-09 05:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:44:59.919501446 +0000 UTC m=+973.613844415" watchObservedRunningTime="2026-06-09 05:44:59.920868135 +0000 UTC m=+973.615211116" Jun 09 05:45:05.013376 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:05.013335 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:45:05.013885 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:05.013440 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:45:05.026138 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:05.026103 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:45:05.943248 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:05.943222 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5" Jun 09 05:45:12.964701 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:12.964665 2567 generic.go:356] "Generic (PLEG): container finished" podID="c626053c-b8ea-4714-afed-2ab6a9b12901" containerID="59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65" exitCode=1 Jun 09 05:45:12.965126 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:12.964732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerDied","Data":"59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65"} Jun 09 05:45:12.965126 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:12.964793 2567 scope.go:117] "RemoveContainer" containerID="b22e5c08cb8e0d0d18b668b39a18ed42faca59d82c37f1dd034e46c41486d54c" Jun 09 05:45:12.965126 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:12.965115 2567 scope.go:117] "RemoveContainer" containerID="59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65" Jun 09 05:45:12.965407 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:45:12.965370 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:45:14.392627 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:14.392591 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:45:14.393068 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:14.392876 2567 scope.go:117] "RemoveContainer" containerID="59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65" Jun 09 05:45:14.393068 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:45:14.393058 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:45:25.793849 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:25.793820 2567 scope.go:117] "RemoveContainer" containerID="59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65" Jun 09 05:45:26.032833 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:26.032799 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7"} Jun 09 05:45:26.033013 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:26.032996 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:45:37.039017 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:37.038986 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:45:47.116113 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.116051 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:45:47.221133 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.221097 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cbcff6cff-xswxm"] Jun 09 05:45:47.221409 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.221343 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" podUID="01a9d0e9-e88f-4ec4-b672-705434fe63e1" containerName="manager" containerID="cri-o://b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20" gracePeriod=30 Jun 09 05:45:47.487982 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.487956 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:45:47.501168 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.501145 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01a9d0e9-e88f-4ec4-b672-705434fe63e1-webhook-cert\") pod \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " Jun 09 05:45:47.501270 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.501241 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql8qf\" (UniqueName: \"kubernetes.io/projected/01a9d0e9-e88f-4ec4-b672-705434fe63e1-kube-api-access-ql8qf\") pod \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\" (UID: \"01a9d0e9-e88f-4ec4-b672-705434fe63e1\") " Jun 09 05:45:47.503349 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.503327 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a9d0e9-e88f-4ec4-b672-705434fe63e1-kube-api-access-ql8qf" (OuterVolumeSpecName: "kube-api-access-ql8qf") pod "01a9d0e9-e88f-4ec4-b672-705434fe63e1" (UID: "01a9d0e9-e88f-4ec4-b672-705434fe63e1"). InnerVolumeSpecName "kube-api-access-ql8qf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 05:45:47.503770 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.503745 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a9d0e9-e88f-4ec4-b672-705434fe63e1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "01a9d0e9-e88f-4ec4-b672-705434fe63e1" (UID: "01a9d0e9-e88f-4ec4-b672-705434fe63e1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 05:45:47.602586 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.602533 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01a9d0e9-e88f-4ec4-b672-705434fe63e1-webhook-cert\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:45:47.602586 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:47.602580 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql8qf\" (UniqueName: \"kubernetes.io/projected/01a9d0e9-e88f-4ec4-b672-705434fe63e1-kube-api-access-ql8qf\") on node \"ip-10-0-135-121.ec2.internal\" DevicePath \"\"" Jun 09 05:45:48.141207 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.141167 2567 generic.go:356] "Generic (PLEG): container finished" podID="01a9d0e9-e88f-4ec4-b672-705434fe63e1" containerID="b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20" exitCode=0 Jun 09 05:45:48.141207 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.141199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" event={"ID":"01a9d0e9-e88f-4ec4-b672-705434fe63e1","Type":"ContainerDied","Data":"b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20"} Jun 09 05:45:48.141777 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.141217 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" Jun 09 05:45:48.141777 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.141262 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cbcff6cff-xswxm" event={"ID":"01a9d0e9-e88f-4ec4-b672-705434fe63e1","Type":"ContainerDied","Data":"5b2deaa40d51f6d62090dc8f32934c85ca646a47135914d336f02006e2845289"} Jun 09 05:45:48.141777 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.141288 2567 scope.go:117] "RemoveContainer" containerID="b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20" Jun 09 05:45:48.150371 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.150353 2567 scope.go:117] "RemoveContainer" containerID="b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20" Jun 09 05:45:48.150675 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:45:48.150655 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20\": container with ID starting with b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20 not found: ID does not exist" containerID="b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20" Jun 09 05:45:48.150727 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.150683 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20"} err="failed to get container status \"b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20\": rpc error: code = NotFound desc = could not find container \"b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20\": container with ID starting with b399229b92ca01acac747589c032527491bba8c2fc04015e5ea0873a4d32bf20 not found: ID does not exist" Jun 09 05:45:48.163607 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.163586 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cbcff6cff-xswxm"] Jun 09 05:45:48.167124 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.167103 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cbcff6cff-xswxm"] Jun 09 05:45:48.798793 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.798762 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a9d0e9-e88f-4ec4-b672-705434fe63e1" path="/var/lib/kubelet/pods/01a9d0e9-e88f-4ec4-b672-705434fe63e1/volumes" Jun 09 05:45:48.823079 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:48.823051 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:45:58.571613 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:45:58.571576 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:46:19.264053 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:46:19.264013 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:46:31.079686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:46:31.079647 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:46:31.196317 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:46:31.196279 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:46:39.578964 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:46:39.578855 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:46:45.835443 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:46:45.835378 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:46:46.154117 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:46:46.154078 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:18.024312 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:18.024278 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:18.131650 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:18.131612 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:26.614145 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:26.614114 2567 generic.go:356] "Generic (PLEG): container finished" podID="c626053c-b8ea-4714-afed-2ab6a9b12901" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" exitCode=1 Jun 09 05:47:26.614609 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:26.614193 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerDied","Data":"9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7"} Jun 09 05:47:26.614609 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:26.614257 2567 scope.go:117] "RemoveContainer" containerID="59ce21cc6c8cd14a16eff2cdd6766b4cf4e940606766915434b245ac09cbec65" Jun 09 05:47:26.614609 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:26.614592 2567 scope.go:117] "RemoveContainer" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" Jun 09 05:47:26.614854 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:47:26.614833 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:47:26.685303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:26.685272 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:27.037526 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:27.037492 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:47:27.620725 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:27.620700 2567 scope.go:117] "RemoveContainer" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" Jun 09 05:47:27.621101 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:47:27.620885 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:47:30.278695 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:30.278664 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:47:30.279059 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:30.278999 2567 scope.go:117] "RemoveContainer" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" Jun 09 05:47:30.279194 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:47:30.279173 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:47:41.793367 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:41.793333 2567 scope.go:117] "RemoveContainer" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" Jun 09 05:47:41.793763 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:47:41.793580 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:47:42.035088 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:42.035035 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:47.796154 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:47.796106 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:47.900276 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:47.900218 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:48.010570 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:48.010521 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:48.174431 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:48.174374 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:48.319262 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:48.319228 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:47:56.795947 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:56.795916 2567 scope.go:117] "RemoveContainer" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" Jun 09 05:47:57.764965 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:57.764931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d"} Jun 09 05:47:57.765158 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:47:57.765137 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:48:04.658302 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:04.658264 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:04.769785 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:04.769751 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:04.874354 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:04.874321 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:08.771121 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:08.771043 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:48:10.727967 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:10.727934 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:10.837363 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:10.837318 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:18.861298 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:18.861263 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:18.963131 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:18.963091 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:19.061341 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:19.061307 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:19.160316 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:19.160289 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:35.549303 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:35.549263 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:35.651421 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:35.651348 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:35.756538 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:35.756480 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:41.511751 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:41.511715 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:41.611926 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:41.611896 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:48:46.831079 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:46.831051 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:48:46.833936 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:48:46.833915 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:49:01.852986 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:01.852945 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:01.961894 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:01.961854 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:02.064536 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:02.064503 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:07.795841 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:07.795796 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:07.851593 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:07.851559 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:07.913191 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:07.913155 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:08.013884 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:08.013851 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:08.125135 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:08.125074 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:13.755023 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:13.754987 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:13.855198 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:13.855164 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:49:57.319821 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:57.319730 2567 generic.go:356] "Generic (PLEG): container finished" podID="c626053c-b8ea-4714-afed-2ab6a9b12901" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" exitCode=1 Jun 09 05:49:57.319821 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:57.319763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerDied","Data":"cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d"} Jun 09 05:49:57.320333 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:57.319830 2567 scope.go:117] "RemoveContainer" containerID="9e28968972a642cef18f1a1c728432d04808ab1f45253231aad41dd7e5512aa7" Jun 09 05:49:57.320333 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:57.320167 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:49:57.320440 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:49:57.320380 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:49:58.769503 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:58.769467 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:49:58.769889 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:49:58.769819 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:49:58.770000 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:49:58.769981 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:50:10.277780 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:10.277742 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:50:10.278181 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:10.278106 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:50:10.278288 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:50:10.278268 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:50:23.793491 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:23.793460 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:50:23.793881 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:50:23.793672 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:50:36.795765 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:36.795732 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:50:36.796279 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:50:36.795938 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:50:48.450461 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:48.450421 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:50:49.469986 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:49.469949 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:50:49.793752 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:49.793676 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:50:50.584956 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:50.584924 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77"} Jun 09 05:50:50.591946 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:50:50.586105 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:51:01.591034 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:51:01.591002 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:51:43.205086 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:51:43.205038 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:51:43.259500 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:51:43.259462 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:52:39.047637 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:39.047551 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:52:39.146610 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:39.146576 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:52:50.151429 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:50.151332 2567 generic.go:356] "Generic (PLEG): container finished" podID="c626053c-b8ea-4714-afed-2ab6a9b12901" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" exitCode=1 Jun 09 05:52:50.151429 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:50.151410 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerDied","Data":"320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77"} Jun 09 05:52:50.151938 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:50.151468 2567 scope.go:117] "RemoveContainer" containerID="cec580749cebd9c9fed0ccb9a12e8a164d7cb2df07adb76f4dfa8c5d55857e6d" Jun 09 05:52:50.151938 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:50.151761 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:52:50.152024 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:52:50.151960 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:52:50.278490 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:50.278459 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:52:51.157497 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:51.157469 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:52:51.157974 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:52:51.157672 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:52:51.590290 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:51.590210 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:52:52.161855 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:52:52.161826 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:52:52.162260 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:52:52.162012 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:53:03.793794 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:03.793763 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:53:03.794188 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:53:03.793954 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:53:14.799885 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:14.799848 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:53:14.800367 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:53:14.800098 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:53:29.793424 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:29.793371 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:53:29.793898 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:53:29.793614 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:53:41.793432 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:41.793376 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:53:41.793825 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:53:41.793593 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:53:46.860400 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:46.860353 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:53:46.864675 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:46.864653 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:53:52.793827 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:53:52.793793 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:53:52.794234 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:53:52.793990 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:54:02.866523 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:02.866478 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-76cdcb8cdb-6dvt9_f7f81a0b-8dc9-446f-8445-7216ba7df95b/maas-api/0.log" Jun 09 05:54:03.664697 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:03.664660 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66ccd5c574-zmwk2_ce1caaf1-a826-4e7a-8710-5467bcdd255f/copy-manifests/0.log" Jun 09 05:54:03.672411 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:03.672361 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66ccd5c574-zmwk2_ce1caaf1-a826-4e7a-8710-5467bcdd255f/manager/0.log" Jun 09 05:54:04.793898 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:04.793865 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:54:04.794278 ip-10-0-135-121 kubenswrapper[2567]: E0609 05:54:04.794094 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-mrvh5_openshift-ingress(c626053c-b8ea-4714-afed-2ab6a9b12901)\"" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" podUID="c626053c-b8ea-4714-afed-2ab6a9b12901" Jun 09 05:54:05.363196 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:05.363165 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-v6krc_826c9a71-f354-4554-bbdb-d561b811b26c/kuadrant-console-plugin/0.log" Jun 09 05:54:05.818164 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:05.818127 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-d2s2q_f428f7c2-d4e8-422a-b08b-b883dccfb7f1/manager/0.log" Jun 09 05:54:06.478826 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:06.478723 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85cbb6b997-hm92g_9ac14b4b-1843-4403-a3e0-64a6a620578c/kube-auth-proxy/0.log" Jun 09 05:54:06.707444 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:06.707412 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-jjbkm_4e0439f0-07d6-42db-9d09-d36be3a708e6/payload-pre-processing/0.log" Jun 09 05:54:06.821105 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:06.821020 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/4.log" Jun 09 05:54:06.931208 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:06.931178 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4bdb859-fr94v_433e22e2-9c9b-4542-988b-9572b5ad189f/router/0.log" Jun 09 05:54:07.282994 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.282962 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj_c1dbcd90-9733-44e2-a67c-3a8c950348c0/main/0.log" Jun 09 05:54:07.292033 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.292004 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-kjgpj_c1dbcd90-9733-44e2-a67c-3a8c950348c0/storage-initializer/0.log" Jun 09 05:54:07.401695 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.401665 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-sd6tb_cd80f705-65ed-4264-8e56-2d2824c78c20/storage-initializer/0.log" Jun 09 05:54:07.409053 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.409028 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-sd6tb_cd80f705-65ed-4264-8e56-2d2824c78c20/main/0.log" Jun 09 05:54:07.519343 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.519282 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk_adc808e0-58a1-464c-94a8-ece0b85438fa/storage-initializer/0.log" Jun 09 05:54:07.527063 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.527040 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-fpdwk_adc808e0-58a1-464c-94a8-ece0b85438fa/main/0.log" Jun 09 05:54:07.637083 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.637055 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5_7f2b84cf-96be-4d6f-a701-af28374f22b6/storage-initializer/0.log" Jun 09 05:54:07.645234 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.645210 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477chdkr5_7f2b84cf-96be-4d6f-a701-af28374f22b6/main/0.log" Jun 09 05:54:07.757903 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.757874 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l_c50027c5-3f58-4001-bfcf-2a50462860f6/storage-initializer/0.log" Jun 09 05:54:07.766055 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:07.766024 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-gvx7l_c50027c5-3f58-4001-bfcf-2a50462860f6/main/0.log" Jun 09 05:54:12.169724 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:12.169687 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:54:12.275475 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:12.275439 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:54:15.170441 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:15.170410 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-28hwr_345979f3-c948-4fc6-87e0-b7e7ac3242e5/global-pull-secret-syncer/0.log" Jun 09 05:54:15.309102 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:15.309070 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" Jun 09 05:54:15.440949 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:15.440861 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-121.ec2.internal_7bbdc86751d54a40c4e45022ba51be3b/haproxy/0.log" Jun 09 05:54:16.795603 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:16.795576 2567 scope.go:117] "RemoveContainer" containerID="320ffa3d63198551cdd7343de71876c891b20a37b60b36bc1046503c766c8e77" Jun 09 05:54:17.569120 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:17.569086 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" event={"ID":"c626053c-b8ea-4714-afed-2ab6a9b12901","Type":"ContainerStarted","Data":"df17d22b489c3c39e48eb07cabad5736f9290b87134cbde904b1f1dd02e9c9aa"} Jun 09 05:54:17.569313 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:17.569296 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:54:20.057118 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:20.057039 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-v6krc_826c9a71-f354-4554-bbdb-d561b811b26c/kuadrant-console-plugin/0.log" Jun 09 05:54:20.253770 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:20.253737 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-d2s2q_f428f7c2-d4e8-422a-b08b-b883dccfb7f1/manager/0.log" Jun 09 05:54:22.328983 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.328903 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-x592k_e0e540f5-fe13-48bc-9c18-84ed0ef1dae7/monitoring-plugin/0.log" Jun 09 05:54:22.527177 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.527147 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvfb2_fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c/node-exporter/0.log" Jun 09 05:54:22.550358 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.550324 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvfb2_fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c/kube-rbac-proxy/0.log" Jun 09 05:54:22.581170 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.581107 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvfb2_fbc5ebf5-bcbc-4e18-9110-d1c2b33e965c/init-textfile/0.log" Jun 09 05:54:22.706539 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.706511 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/prometheus/0.log" Jun 09 05:54:22.728132 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.728099 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/config-reloader/0.log" Jun 09 05:54:22.748841 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.748812 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/thanos-sidecar/0.log" Jun 09 05:54:22.770029 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.770003 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/kube-rbac-proxy-web/0.log" Jun 09 05:54:22.791359 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.791319 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/kube-rbac-proxy/0.log" Jun 09 05:54:22.813380 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.813341 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/kube-rbac-proxy-thanos/0.log" Jun 09 05:54:22.863669 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:22.863589 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e83ed02-1542-4b4e-9af8-fe32c2f8922e/init-config-reloader/0.log" Jun 09 05:54:23.104976 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.104951 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b676678c7-l7dkh_6dff0626-9f48-4d33-b8da-77f7c9b29e27/thanos-query/0.log" Jun 09 05:54:23.134837 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.134815 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b676678c7-l7dkh_6dff0626-9f48-4d33-b8da-77f7c9b29e27/kube-rbac-proxy-web/0.log" Jun 09 05:54:23.161054 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.161029 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b676678c7-l7dkh_6dff0626-9f48-4d33-b8da-77f7c9b29e27/kube-rbac-proxy/0.log" Jun 09 05:54:23.181649 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.181612 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4j88l_f3d44367-adf6-4045-b571-8936e08e952b/konnectivity-agent/0.log" limit=1024 Jun 09 05:54:23.189727 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.189706 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b676678c7-l7dkh_6dff0626-9f48-4d33-b8da-77f7c9b29e27/prom-label-proxy/0.log" Jun 09 05:54:23.216043 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.216018 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b676678c7-l7dkh_6dff0626-9f48-4d33-b8da-77f7c9b29e27/kube-rbac-proxy-rules/0.log" Jun 09 05:54:23.242074 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:23.242051 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b676678c7-l7dkh_6dff0626-9f48-4d33-b8da-77f7c9b29e27/kube-rbac-proxy-metrics/0.log" Jun 09 05:54:25.357258 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:25.357221 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/1.log" Jun 09 05:54:25.367266 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:25.367240 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-t4t2t_8850590e-542e-45ca-a2a7-8ed744959331/console-operator/2.log" Jun 09 05:54:26.025675 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.025649 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-kpcfx_cfc3eea4-b827-41a8-b62d-3ce6fc89ef64/download-server/0.log" Jun 09 05:54:26.132138 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.132098 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l"] Jun 09 05:54:26.132576 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.132558 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a9d0e9-e88f-4ec4-b672-705434fe63e1" containerName="manager" Jun 09 05:54:26.132686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.132578 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a9d0e9-e88f-4ec4-b672-705434fe63e1" containerName="manager" Jun 09 05:54:26.132686 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.132671 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a9d0e9-e88f-4ec4-b672-705434fe63e1" containerName="manager" Jun 09 05:54:26.135860 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.135837 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.138348 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.138327 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4628p\"/\"kube-root-ca.crt\"" Jun 09 05:54:26.138626 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.138604 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4628p\"/\"openshift-service-ca.crt\"" Jun 09 05:54:26.138708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.138692 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4628p\"/\"default-dockercfg-pnvwv\"" Jun 09 05:54:26.147653 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.147630 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l"] Jun 09 05:54:26.273774 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.273743 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-sys\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.273774 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.273773 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-podres\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.273965 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.273811 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-proc\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.273965 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.273840 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h57tx\" (UniqueName: \"kubernetes.io/projected/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-kube-api-access-h57tx\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.273965 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.273888 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-lib-modules\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374422 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374368 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-lib-modules\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374452 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-sys\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374470 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-podres\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374503 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-proc\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h57tx\" (UniqueName: \"kubernetes.io/projected/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-kube-api-access-h57tx\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374539 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-lib-modules\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374583 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-sys\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374600 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-proc\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.374818 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.374600 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-podres\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.383163 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.383134 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h57tx\" (UniqueName: \"kubernetes.io/projected/c362548d-a6b5-4e9e-a03c-ce1cb4cb488f-kube-api-access-h57tx\") pod \"perf-node-gather-daemonset-qb42l\" (UID: \"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f\") " pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.446300 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.446261 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:26.574535 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.574512 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l"] Jun 09 05:54:26.576506 ip-10-0-135-121 kubenswrapper[2567]: W0609 05:54:26.576473 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc362548d_a6b5_4e9e_a03c_ce1cb4cb488f.slice/crio-4b5c93d2729648f98bae5455e1daac43e272fc7db50711a47cb8d5956d77dac9 WatchSource:0}: Error finding container 4b5c93d2729648f98bae5455e1daac43e272fc7db50711a47cb8d5956d77dac9: Status 404 returned error can't find the container with id 4b5c93d2729648f98bae5455e1daac43e272fc7db50711a47cb8d5956d77dac9 Jun 09 05:54:26.612141 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:26.612108 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" event={"ID":"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f","Type":"ContainerStarted","Data":"4b5c93d2729648f98bae5455e1daac43e272fc7db50711a47cb8d5956d77dac9"} Jun 09 05:54:27.621263 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:27.621227 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" event={"ID":"c362548d-a6b5-4e9e-a03c-ce1cb4cb488f","Type":"ContainerStarted","Data":"e989c11ba7961571fb14207faaef9aa4e16940bdf203a01be6131beae2ca0e45"} Jun 09 05:54:27.621677 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:27.621325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:27.637291 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:27.637241 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" podStartSLOduration=1.6372281100000001 podStartE2EDuration="1.63722811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 05:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 05:54:27.636336041 +0000 UTC m=+1541.330679033" watchObservedRunningTime="2026-06-09 05:54:27.63722811 +0000 UTC m=+1541.331571089" Jun 09 05:54:27.998112 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:27.998030 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8r8dx_40108820-3325-4e88-a8b1-056266249a9b/dns/0.log" Jun 09 05:54:28.019274 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:28.019241 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8r8dx_40108820-3325-4e88-a8b1-056266249a9b/kube-rbac-proxy/0.log" Jun 09 05:54:28.139974 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:28.139942 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l6qxp_1b6e3541-7a83-41ea-bdea-f8e2f4787005/dns-node-resolver/0.log" Jun 09 05:54:28.575580 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:28.575550 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mrvh5" Jun 09 05:54:28.783225 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:28.783200 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vqxk4_a5af69f3-c1ad-4531-b392-4313706bcb16/node-ca/0.log" Jun 09 05:54:29.834368 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:29.834338 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85cbb6b997-hm92g_9ac14b4b-1843-4403-a3e0-64a6a620578c/kube-auth-proxy/0.log" Jun 09 05:54:29.928857 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:29.928825 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-jjbkm_4e0439f0-07d6-42db-9d09-d36be3a708e6/payload-pre-processing/0.log" Jun 09 05:54:29.955159 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:29.955130 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/5.log" Jun 09 05:54:29.959281 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:29.959255 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/4.log" Jun 09 05:54:29.982814 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:29.982794 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4bdb859-fr94v_433e22e2-9c9b-4542-988b-9572b5ad189f/router/0.log" Jun 09 05:54:30.614593 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:30.614549 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dhhmg_c9a9a461-a7dc-4aef-9008-e2418f03da29/serve-healthcheck-canary/0.log" Jun 09 05:54:31.170714 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:31.170681 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b5g88_e92aa975-0d99-4d99-bee0-a1e94cef975f/kube-rbac-proxy/0.log" Jun 09 05:54:31.190810 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:31.190788 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b5g88_e92aa975-0d99-4d99-bee0-a1e94cef975f/exporter/0.log" Jun 09 05:54:31.213978 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:31.213950 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b5g88_e92aa975-0d99-4d99-bee0-a1e94cef975f/extractor/0.log" Jun 09 05:54:33.638357 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:33.638325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4628p/perf-node-gather-daemonset-qb42l" Jun 09 05:54:33.666167 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:33.666137 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-76cdcb8cdb-6dvt9_f7f81a0b-8dc9-446f-8445-7216ba7df95b/maas-api/0.log" Jun 09 05:54:33.999184 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:33.999099 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66ccd5c574-zmwk2_ce1caaf1-a826-4e7a-8710-5467bcdd255f/manager/0.log" Jun 09 05:54:34.050771 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:34.050746 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66ccd5c574-zmwk2_ce1caaf1-a826-4e7a-8710-5467bcdd255f/copy-manifests/0.log" Jun 09 05:54:35.651301 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:35.651270 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55f7df5cdd-qk9g4_bbc854c2-f5dd-4efa-b7ef-2e377c2a7072/manager/0.log" Jun 09 05:54:40.618164 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:40.618124 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85cbb6b997-hm92g_9ac14b4b-1843-4403-a3e0-64a6a620578c/kube-auth-proxy/0.log" Jun 09 05:54:40.704049 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:40.704014 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-jjbkm_4e0439f0-07d6-42db-9d09-d36be3a708e6/payload-pre-processing/0.log" Jun 09 05:54:40.715421 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:40.715397 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/5.log" Jun 09 05:54:40.719451 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:40.719428 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/4.log" Jun 09 05:54:40.733473 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:40.733452 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4bdb859-fr94v_433e22e2-9c9b-4542-988b-9572b5ad189f/router/0.log" Jun 09 05:54:41.029943 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.029868 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-bs8kx_a71c6eb8-6b46-4747-bf51-9e38450fa9df/migrator/0.log" Jun 09 05:54:41.050843 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.050808 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-bs8kx_a71c6eb8-6b46-4747-bf51-9e38450fa9df/graceful-termination/0.log" Jun 09 05:54:41.873377 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.873346 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85cbb6b997-hm92g_9ac14b4b-1843-4403-a3e0-64a6a620578c/kube-auth-proxy/0.log" Jun 09 05:54:41.960777 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.960718 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-jjbkm_4e0439f0-07d6-42db-9d09-d36be3a708e6/payload-pre-processing/0.log" Jun 09 05:54:41.973867 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.973819 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/5.log" Jun 09 05:54:41.977852 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.977827 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/4.log" Jun 09 05:54:41.995708 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:41.995682 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4bdb859-fr94v_433e22e2-9c9b-4542-988b-9572b5ad189f/router/0.log" Jun 09 05:54:42.796376 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.796340 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/kube-multus-additional-cni-plugins/0.log" Jun 09 05:54:42.821273 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.821239 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/egress-router-binary-copy/0.log" Jun 09 05:54:42.844134 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.844113 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/cni-plugins/0.log" Jun 09 05:54:42.870720 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.870683 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/bond-cni-plugin/0.log" Jun 09 05:54:42.889864 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.889839 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/routeoverride-cni/0.log" Jun 09 05:54:42.910794 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.910772 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/whereabouts-cni-bincopy/0.log" Jun 09 05:54:42.933661 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:42.933645 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mcm2j_f1896cb4-a0d8-4521-83b7-26400e9f186c/whereabouts-cni/0.log" Jun 09 05:54:43.138438 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.138412 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jlzjg_601e07b5-4e8e-4935-8288-0be80cac7a3b/kube-multus/0.log" Jun 09 05:54:43.212276 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.212252 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wfj72_879b1a42-fd63-4e8a-be9b-7886e17a8408/network-metrics-daemon/0.log" Jun 09 05:54:43.235108 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.235083 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wfj72_879b1a42-fd63-4e8a-be9b-7886e17a8408/kube-rbac-proxy/0.log" Jun 09 05:54:43.574901 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.574821 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85cbb6b997-hm92g_9ac14b4b-1843-4403-a3e0-64a6a620578c/kube-auth-proxy/0.log" Jun 09 05:54:43.654710 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.654672 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-jjbkm_4e0439f0-07d6-42db-9d09-d36be3a708e6/payload-pre-processing/0.log" Jun 09 05:54:43.666754 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.666728 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/5.log" Jun 09 05:54:43.670807 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.670787 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mrvh5_c626053c-b8ea-4714-afed-2ab6a9b12901/payload-processing/4.log" Jun 09 05:54:43.683098 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:43.683080 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4bdb859-fr94v_433e22e2-9c9b-4542-988b-9572b5ad189f/router/0.log" Jun 09 05:54:44.630941 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.630914 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/ovn-controller/0.log" Jun 09 05:54:44.653470 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.653445 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/ovn-acl-logging/0.log" Jun 09 05:54:44.682237 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.682217 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/kube-rbac-proxy-node/0.log" Jun 09 05:54:44.706580 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.706558 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/kube-rbac-proxy-ovn-metrics/0.log" Jun 09 05:54:44.721860 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.721840 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/northd/0.log" Jun 09 05:54:44.742200 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.742178 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/nbdb/0.log" Jun 09 05:54:44.764773 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.764756 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/sbdb/0.log" Jun 09 05:54:44.830465 ip-10-0-135-121 kubenswrapper[2567]: I0609 05:54:44.830437 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jspp7_84f4ae1c-7460-4272-a8ed-ca8cc207ac56/ovnkube-controller/0.log"