Jun 05 13:03:43.077453 ip-10-0-141-104 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 05 13:03:43.077466 ip-10-0-141-104 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 05 13:03:43.077476 ip-10-0-141-104 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 05 13:03:43.077673 ip-10-0-141-104 systemd[1]: Failed to start Kubernetes Kubelet. Jun 05 13:03:53.105122 ip-10-0-141-104 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 05 13:03:53.105140 ip-10-0-141-104 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 04769b170e47447c9786a880949b4e02 -- Jun 05 13:06:27.245557 ip-10-0-141-104 systemd[1]: Starting Kubernetes Kubelet... Jun 05 13:06:27.632799 ip-10-0-141-104 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 05 13:06:27.632799 ip-10-0-141-104 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 05 13:06:27.632799 ip-10-0-141-104 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 05 13:06:27.632799 ip-10-0-141-104 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 05 13:06:27.632799 ip-10-0-141-104 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 05 13:06:27.633836 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.633702 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 05 13:06:27.637833 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637817 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 05 13:06:27.637833 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637831 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637836 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637840 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637843 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637848 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637851 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637854 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637857 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637860 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637863 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637865 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637868 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637870 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637878 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637881 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637884 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637887 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637889 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637892 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 05 13:06:27.637892 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637895 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637898 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637901 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637904 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637907 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637909 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637912 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637914 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637917 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637920 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637922 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637924 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637927 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637929 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637931 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637934 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637937 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637939 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637943 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637945 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 05 13:06:27.638345 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637948 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637950 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637952 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637955 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637957 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637960 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637962 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637964 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637967 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637969 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637971 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637974 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637976 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637979 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637982 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637984 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637987 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637991 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637993 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637996 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 05 13:06:27.638822 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.637999 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638002 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638004 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638006 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638009 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638011 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638014 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638016 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638019 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638021 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638023 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638028 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638032 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638034 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638037 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638039 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638042 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638045 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638047 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 05 13:06:27.639322 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638050 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638052 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638056 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638059 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638061 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638064 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638066 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638070 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638073 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638077 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638080 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638083 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638086 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638089 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638091 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638094 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638096 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638099 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638101 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 05 13:06:27.639765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638104 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638783 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638789 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638792 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638795 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638798 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638801 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638804 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638806 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638809 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638813 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638817 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638820 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638822 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638825 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638828 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638831 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638833 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638836 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 05 13:06:27.640210 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638839 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638842 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638845 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638847 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638850 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638852 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638855 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638857 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638860 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638862 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638865 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638867 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638870 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638872 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638875 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638877 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638880 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638882 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638884 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638887 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 05 13:06:27.640677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638894 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638897 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638900 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638903 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638905 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638907 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638910 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638912 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638915 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638917 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638920 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638923 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638927 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638933 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638936 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638939 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638942 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638945 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638948 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 05 13:06:27.641162 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638950 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638953 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638956 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638958 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638961 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638963 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638966 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638968 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638971 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638973 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638976 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638978 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638981 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638983 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638986 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638989 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638991 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638993 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.638997 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639000 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 05 13:06:27.641637 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639002 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639004 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639007 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639009 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639012 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639015 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639018 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639020 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639023 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639025 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639028 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639030 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639033 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639035 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639037 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639040 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639042 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639045 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639047 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639050 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 05 13:06:27.642118 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639052 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639055 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639448 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639453 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639456 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639459 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639462 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639465 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639467 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639471 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639474 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639477 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639481 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639485 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639488 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639492 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639496 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639499 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639501 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 05 13:06:27.642629 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639504 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639507 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639510 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639513 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639515 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639518 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639521 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639524 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639527 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639529 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639532 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639534 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639536 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639539 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639541 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639544 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639546 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639549 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639552 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639554 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 05 13:06:27.643088 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639562 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639565 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639567 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639570 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639572 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639575 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639578 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639580 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639582 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639585 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639587 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639590 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639592 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639594 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639597 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639599 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639601 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639604 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639606 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639609 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 05 13:06:27.643582 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639611 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639614 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639616 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639618 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639621 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639623 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639626 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639628 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639630 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639633 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639635 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639637 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639640 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639642 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639647 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639653 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639656 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639658 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639661 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639665 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 05 13:06:27.644089 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639668 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639671 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639674 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639676 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639679 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639682 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639684 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639687 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639689 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639691 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639694 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639696 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639699 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639701 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639703 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639706 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639709 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639711 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639714 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639716 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 05 13:06:27.644579 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639719 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.639721 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639791 2570 flags.go:64] FLAG: --address="0.0.0.0" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639798 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639804 2570 flags.go:64] FLAG: --anonymous-auth="true" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639808 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639815 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639820 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639824 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639829 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639832 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639835 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639838 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639841 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639844 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639847 2570 flags.go:64] FLAG: --cgroup-root="" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639850 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639853 2570 flags.go:64] FLAG: --client-ca-file="" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639856 2570 flags.go:64] FLAG: --cloud-provider="external" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639859 2570 flags.go:64] FLAG: --cluster-dns="[]" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639863 2570 flags.go:64] FLAG: --cluster-domain="" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639865 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639869 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639871 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 05 13:06:27.645061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639875 2570 flags.go:64] FLAG: --container-log-max-files="5" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639878 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639881 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639884 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639887 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639890 2570 flags.go:64] FLAG: --contention-profiling="false" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639893 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639896 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639899 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639901 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639905 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639908 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639911 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639913 2570 flags.go:64] FLAG: --enable-load-reader="false" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639917 2570 flags.go:64] FLAG: --enable-server="true" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639921 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639926 2570 flags.go:64] FLAG: --event-burst="100" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639929 2570 flags.go:64] FLAG: --event-qps="50" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639932 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639934 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639937 2570 flags.go:64] FLAG: --eviction-hard="" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639941 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639944 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639946 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639950 2570 flags.go:64] FLAG: --eviction-soft="" Jun 05 13:06:27.645726 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639952 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639955 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639958 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639960 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639963 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639966 2570 flags.go:64] FLAG: --fail-swap-on="true" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639969 2570 flags.go:64] FLAG: --feature-gates="" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639972 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639975 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639978 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639981 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639984 2570 flags.go:64] FLAG: --healthz-port="10248" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639987 2570 flags.go:64] FLAG: --help="false" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639990 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639993 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639996 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.639999 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640002 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640005 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640008 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640011 2570 flags.go:64] FLAG: --image-service-endpoint="" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640014 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640018 2570 flags.go:64] FLAG: --kube-api-burst="100" Jun 05 13:06:27.646400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640021 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640024 2570 flags.go:64] FLAG: --kube-api-qps="50" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640027 2570 flags.go:64] FLAG: --kube-reserved="" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640030 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640033 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640035 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640038 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640041 2570 flags.go:64] FLAG: --lock-file="" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640044 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640046 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640050 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640055 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640058 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640061 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640063 2570 flags.go:64] FLAG: --logging-format="text" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640066 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640069 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640072 2570 flags.go:64] FLAG: --manifest-url="" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640075 2570 flags.go:64] FLAG: --manifest-url-header="" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640079 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640082 2570 flags.go:64] FLAG: --max-open-files="1000000" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640085 2570 flags.go:64] FLAG: --max-pods="110" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640088 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640091 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640093 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jun 05 13:06:27.647000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640096 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640099 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640102 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640105 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640112 2570 flags.go:64] FLAG: --node-status-max-images="50" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640115 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640119 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640121 2570 flags.go:64] FLAG: --pod-cidr="" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640125 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640130 2570 flags.go:64] FLAG: --pod-manifest-path="" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640133 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640136 2570 flags.go:64] FLAG: --pods-per-core="0" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640138 2570 flags.go:64] FLAG: --port="10250" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640142 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640146 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-073e6a37690232fc8" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640149 2570 flags.go:64] FLAG: --qos-reserved="" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640152 2570 flags.go:64] FLAG: --read-only-port="10255" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640155 2570 flags.go:64] FLAG: --register-node="true" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640158 2570 flags.go:64] FLAG: --register-with-taints="" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640162 2570 flags.go:64] FLAG: --registry-burst="10" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640164 2570 flags.go:64] FLAG: --registry-qps="5" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640167 2570 flags.go:64] FLAG: --reserved-cpus="" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640170 2570 flags.go:64] FLAG: --reserved-memory="" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640174 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640176 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 05 13:06:27.647602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640179 2570 flags.go:64] FLAG: --rotate-certificates="false" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640182 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640184 2570 flags.go:64] FLAG: --runonce="false" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640187 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640190 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640193 2570 flags.go:64] FLAG: --seccomp-default="false" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640196 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640199 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640201 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640204 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640207 2570 flags.go:64] FLAG: --storage-driver-password="root" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640210 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640213 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640216 2570 flags.go:64] FLAG: --storage-driver-user="root" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640221 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640224 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640227 2570 flags.go:64] FLAG: --system-cgroups="" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640230 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640235 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640238 2570 flags.go:64] FLAG: --tls-cert-file="" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640241 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640260 2570 flags.go:64] FLAG: --tls-min-version="" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640263 2570 flags.go:64] FLAG: --tls-private-key-file="" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640265 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640268 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 05 13:06:27.648223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640271 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640274 2570 flags.go:64] FLAG: --v="2" Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640277 2570 flags.go:64] FLAG: --version="false" Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640281 2570 flags.go:64] FLAG: --vmodule="" Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640285 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.640288 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640404 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640408 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640411 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640414 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640417 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640420 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640422 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640425 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640427 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640430 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640432 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640434 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640437 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640439 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640442 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 05 13:06:27.648884 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640446 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640448 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640451 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640453 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640456 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640458 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640461 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640469 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640473 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640476 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640478 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640481 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640484 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640486 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640489 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640491 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640494 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640496 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640499 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 05 13:06:27.649622 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640501 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640504 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640506 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640509 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640511 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640514 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640516 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640520 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640523 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640526 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640529 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640531 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640534 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640537 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640539 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640542 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640544 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640548 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640550 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640553 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 05 13:06:27.650398 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640557 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640559 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640562 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640564 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640567 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640570 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640573 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640575 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640578 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640580 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640583 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640585 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640587 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640590 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640592 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640595 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640598 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640600 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640603 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640605 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 05 13:06:27.650906 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640608 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640610 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640612 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640615 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640617 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640620 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640636 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640638 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640641 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640643 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640646 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640648 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640652 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640655 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640658 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640660 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640662 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640665 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640667 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640670 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 05 13:06:27.651410 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640672 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640675 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640677 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640680 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.640683 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.641221 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.647539 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.647554 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.648215 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.648225 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648292 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648297 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648301 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648305 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 05 13:06:27.651878 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648308 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648311 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648314 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648317 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648320 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648323 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648325 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648328 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648331 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648333 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648336 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648339 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648342 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648346 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648349 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648352 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648355 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648357 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648360 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 05 13:06:27.652222 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648362 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648364 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648367 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648371 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648374 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648377 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648380 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648383 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648386 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648389 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648391 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648394 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648397 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648400 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648403 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648405 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648408 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648410 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648413 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648415 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 05 13:06:27.652682 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648418 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648420 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648423 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648425 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648428 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648430 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648432 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648435 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648438 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648440 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648442 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648445 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648447 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648449 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648452 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648454 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648457 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648459 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648461 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 05 13:06:27.653171 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648464 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648467 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648469 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648472 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648474 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648477 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648479 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648482 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648484 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648486 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648489 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648491 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648494 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648496 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648499 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648501 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648503 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648506 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648508 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648511 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 05 13:06:27.653724 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648513 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648515 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648518 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648520 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648523 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648525 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648527 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648529 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648532 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648534 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648537 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648539 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648542 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648545 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648547 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648549 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 05 13:06:27.654205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648552 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.648558 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648673 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648678 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648681 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648684 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648687 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648689 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648691 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648694 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648696 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648699 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648701 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648703 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648706 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 05 13:06:27.654592 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648708 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648711 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648713 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648716 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648718 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648720 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648723 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648725 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648727 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648730 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648732 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648734 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648737 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648739 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648742 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648756 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648758 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648761 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648764 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648766 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 05 13:06:27.654958 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648769 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648771 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648773 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648776 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648779 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648781 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648783 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648786 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648788 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648790 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648793 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648795 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648798 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648800 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648803 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648805 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648807 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648810 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648812 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648814 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 05 13:06:27.655442 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648817 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648819 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648822 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648824 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648826 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648829 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648831 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648833 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648836 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648839 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648843 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648847 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648850 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648853 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648856 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648858 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648861 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648863 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648865 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 05 13:06:27.655945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648868 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648870 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648873 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648875 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648877 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648881 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648883 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648886 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648889 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648891 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648894 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648898 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648902 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648904 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648906 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648909 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648912 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648914 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 05 13:06:27.656402 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648918 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648921 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648924 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648927 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648930 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648933 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648936 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648939 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:27.648941 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.648946 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.649107 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.652157 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.653759 2570 server.go:1019] "Starting client certificate rotation" Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.653858 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 05 13:06:27.656830 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.654700 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 05 13:06:27.657164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.656304 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 05 13:06:27.657524 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.657506 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 05 13:06:27.668390 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.668372 2570 log.go:25] "Validated CRI v1 runtime API" Jun 05 13:06:27.671847 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.671829 2570 log.go:25] "Validated CRI v1 image API" Jun 05 13:06:27.675131 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.675114 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 05 13:06:27.677640 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.677616 2570 fs.go:135] Filesystem UUIDs: map[1d8c8a07-64e0-49fb-999c-be8ada84028d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d4dfce9e-9921-4375-802a-d9c43fa487f4:/dev/nvme0n1p4] Jun 05 13:06:27.677718 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.677639 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 05 13:06:27.683279 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.683148 2570 manager.go:216] Machine: {Timestamp:2026-06-05 13:06:27.681848594 +0000 UTC m=+0.323172540 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3201675 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29acf03932abea391a28b2c782e552 SystemUUID:ec29acf0-3932-abea-391a-28b2c782e552 BootID:04769b17-0e47-447c-9786-a880949b4e02 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fe:9b:f6:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fe:9b:f6:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:d8:fe:f7:e0:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 05 13:06:27.683279 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.683271 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 05 13:06:27.683442 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.683372 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 05 13:06:27.686301 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.686279 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 05 13:06:27.686428 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.686306 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-104.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 05 13:06:27.687177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.687167 2570 topology_manager.go:138] "Creating topology manager with none policy" Jun 05 13:06:27.687209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.687179 2570 container_manager_linux.go:309] "Creating device plugin manager" Jun 05 13:06:27.687209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.687187 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 05 13:06:27.687496 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.687481 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 05 13:06:27.687845 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.687834 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 05 13:06:27.688576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.688566 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 05 13:06:27.691564 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691553 2570 state_mem.go:36] "Initialized new in-memory state store" Jun 05 13:06:27.691674 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691665 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 05 13:06:27.691714 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691695 2570 kubelet.go:486] "Attempting to sync node with API server" Jun 05 13:06:27.691714 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691705 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 05 13:06:27.691767 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691720 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 05 13:06:27.691767 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691730 2570 kubelet.go:398] "Adding apiserver pod source" Jun 05 13:06:27.691767 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.691743 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 05 13:06:27.692183 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.692169 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 05 13:06:27.692218 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.692199 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 05 13:06:27.696267 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.696217 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 05 13:06:27.698428 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.698413 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 05 13:06:27.698492 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.698443 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 05 13:06:27.699516 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699506 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699521 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699527 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699532 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699541 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699546 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699556 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699563 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699570 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 05 13:06:27.699571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699575 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 05 13:06:27.699782 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699583 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 05 13:06:27.699782 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.699589 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 05 13:06:27.700263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.700239 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 05 13:06:27.700263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.700263 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 05 13:06:27.703216 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.703204 2570 server.go:1268] "Started kubelet" Jun 05 13:06:27.703338 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.703310 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 05 13:06:27.703410 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.703361 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 05 13:06:27.703464 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.703436 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 05 13:06:27.704702 ip-10-0-141-104 systemd[1]: Started Kubernetes Kubelet. Jun 05 13:06:27.705180 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.705117 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 05 13:06:27.705783 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.705772 2570 server.go:310] "Adding debug handlers to kubelet server" Jun 05 13:06:27.707826 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.707804 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 05 13:06:27.708076 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.708044 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 05 13:06:27.708120 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.708094 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-104.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 05 13:06:27.709578 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.709563 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 05 13:06:27.709670 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.709583 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 05 13:06:27.709670 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.709657 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:27.710303 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710287 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 05 13:06:27.710395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710341 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 05 13:06:27.710395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710359 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 05 13:06:27.710500 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.710416 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 05 13:06:27.710548 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710499 2570 reconstruct.go:97] "Volume reconstruction finished" Jun 05 13:06:27.710548 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710508 2570 reconciler.go:29] "Reconciler: start to sync state" Jun 05 13:06:27.710699 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710680 2570 factory.go:55] Registering systemd factory Jun 05 13:06:27.710699 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710699 2570 factory.go:223] Registration of the systemd container factory successfully Jun 05 13:06:27.711055 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.710924 2570 factory.go:153] Registering CRI-O factory Jun 05 13:06:27.711131 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.711059 2570 factory.go:223] Registration of the crio container factory successfully Jun 05 13:06:27.711179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.711148 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 05 13:06:27.711179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.711174 2570 factory.go:103] Registering Raw factory Jun 05 13:06:27.711294 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.711227 2570 manager.go:1196] Started watching for new ooms in manager Jun 05 13:06:27.711627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.711614 2570 manager.go:349] Starting recovery of all containers Jun 05 13:06:27.713772 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.713682 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-104.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 05 13:06:27.713772 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.713752 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 05 13:06:27.714493 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.713453 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-104.ec2.internal.18b63130a7181adc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-104.ec2.internal,UID:ip-10-0-141-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-104.ec2.internal,},FirstTimestamp:2026-06-05 13:06:27.7031759 +0000 UTC m=+0.344499692,LastTimestamp:2026-06-05 13:06:27.7031759 +0000 UTC m=+0.344499692,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-104.ec2.internal,}" Jun 05 13:06:27.721241 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.721226 2570 manager.go:354] Recovery completed Jun 05 13:06:27.726472 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.726460 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.728157 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728142 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.728227 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728169 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.728227 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728178 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.728676 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728661 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 05 13:06:27.728676 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728676 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 05 13:06:27.728802 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728693 2570 state_mem.go:36] "Initialized new in-memory state store" Jun 05 13:06:27.728802 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.728722 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.729406 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.729347 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.729406 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.729374 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.729406 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.729388 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.729831 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.729819 2570 policy_none.go:49] "None policy: Start" Jun 05 13:06:27.729883 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.729837 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 05 13:06:27.729883 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.729851 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 05 13:06:27.730987 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.730975 2570 policy_none.go:47] "Start" Jun 05 13:06:27.731320 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.731240 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-104.ec2.internal.18b63130a8954895 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-104.ec2.internal,UID:ip-10-0-141-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-104.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-104.ec2.internal,},FirstTimestamp:2026-06-05 13:06:27.728156821 +0000 UTC m=+0.369480612,LastTimestamp:2026-06-05 13:06:27.728156821 +0000 UTC m=+0.369480612,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-104.ec2.internal,}" Jun 05 13:06:27.733439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.733422 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s2sr4" Jun 05 13:06:27.741431 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.741372 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-104.ec2.internal.18b63130a895889c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-104.ec2.internal,UID:ip-10-0-141-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-104.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-104.ec2.internal,},FirstTimestamp:2026-06-05 13:06:27.728173212 +0000 UTC m=+0.369497003,LastTimestamp:2026-06-05 13:06:27.728173212 +0000 UTC m=+0.369497003,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-104.ec2.internal,}" Jun 05 13:06:27.744183 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.744120 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-104.ec2.internal.18b63130a895ace4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-104.ec2.internal,UID:ip-10-0-141-104.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-104.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-104.ec2.internal,},FirstTimestamp:2026-06-05 13:06:27.7281825 +0000 UTC m=+0.369506290,LastTimestamp:2026-06-05 13:06:27.7281825 +0000 UTC m=+0.369506290,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-104.ec2.internal,}" Jun 05 13:06:27.745906 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.745892 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s2sr4" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766592 2570 manager.go:337] "Starting Device Plugin manager" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.766615 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766624 2570 server.go:87] "Starting device plugin registration server" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766822 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766831 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766919 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766987 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.766995 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.767557 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.767591 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:27.775126 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.774894 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 05 13:06:27.775987 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.775969 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 05 13:06:27.776074 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.776007 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 05 13:06:27.776074 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.776025 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jun 05 13:06:27.776074 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.776065 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 05 13:06:27.778793 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.778771 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:27.867818 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.867779 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.868897 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.868884 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.868973 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.868910 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.868973 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.868922 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.868973 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.868951 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.876475 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.876441 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal"] Jun 05 13:06:27.876532 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.876522 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.877242 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.877220 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.877314 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.877270 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.877314 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.877288 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.877400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.877388 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.877432 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.877408 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-104.ec2.internal\": node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:27.878404 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.878391 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.878525 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.878510 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.878558 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.878541 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.879010 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.878995 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.879010 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.879004 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.879171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.879022 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.879171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.879024 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.879171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.879064 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.879171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.879037 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.880207 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.880191 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.880306 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.880219 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 05 13:06:27.880854 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.880838 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientMemory" Jun 05 13:06:27.880933 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.880868 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasNoDiskPressure" Jun 05 13:06:27.880933 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.880881 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeHasSufficientPID" Jun 05 13:06:27.890541 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.890527 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:27.908456 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.908444 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-104.ec2.internal\" not found" node="ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.910698 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.910682 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5808d78f0aee0231302b30ce2d3d649-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal\" (UID: \"c5808d78f0aee0231302b30ce2d3d649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.910774 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.910709 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24f75fd8523075e78d67ffee29b41dde-config\") pod \"kube-apiserver-proxy-ip-10-0-141-104.ec2.internal\" (UID: \"24f75fd8523075e78d67ffee29b41dde\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.910774 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:27.910731 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c5808d78f0aee0231302b30ce2d3d649-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal\" (UID: \"c5808d78f0aee0231302b30ce2d3d649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.912601 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.912585 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-104.ec2.internal\" not found" node="ip-10-0-141-104.ec2.internal" Jun 05 13:06:27.991107 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:27.991089 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:28.011645 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.011626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c5808d78f0aee0231302b30ce2d3d649-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal\" (UID: \"c5808d78f0aee0231302b30ce2d3d649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.011730 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.011654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5808d78f0aee0231302b30ce2d3d649-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal\" (UID: \"c5808d78f0aee0231302b30ce2d3d649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.011730 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.011679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24f75fd8523075e78d67ffee29b41dde-config\") pod \"kube-apiserver-proxy-ip-10-0-141-104.ec2.internal\" (UID: \"24f75fd8523075e78d67ffee29b41dde\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.011821 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.011727 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24f75fd8523075e78d67ffee29b41dde-config\") pod \"kube-apiserver-proxy-ip-10-0-141-104.ec2.internal\" (UID: \"24f75fd8523075e78d67ffee29b41dde\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.011821 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.011728 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c5808d78f0aee0231302b30ce2d3d649-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal\" (UID: \"c5808d78f0aee0231302b30ce2d3d649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.011821 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.011729 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5808d78f0aee0231302b30ce2d3d649-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal\" (UID: \"c5808d78f0aee0231302b30ce2d3d649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.091376 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.091355 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:28.192382 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.192329 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:28.212697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.212680 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.215919 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.215893 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.292508 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.292462 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:28.392924 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.392899 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:28.493496 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.493418 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-104.ec2.internal\" not found" Jun 05 13:06:28.561519 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.561492 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:28.610236 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.610214 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.618985 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.618962 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:28.621737 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.621717 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 05 13:06:28.623153 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.623138 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" Jun 05 13:06:28.631828 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.631802 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 05 13:06:28.652910 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.652891 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 05 13:06:28.653531 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.653000 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 05 13:06:28.653531 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.653041 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 05 13:06:28.653531 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.653045 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 05 13:06:28.676876 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.676858 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:28.692805 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.692786 2570 apiserver.go:52] "Watching apiserver" Jun 05 13:06:28.699606 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.699294 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 05 13:06:28.700182 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.700160 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7q58h","kube-system/konnectivity-agent-5r9xd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59","openshift-cluster-node-tuning-operator/tuned-fthpr","openshift-image-registry/node-ca-zg2xk","openshift-multus/network-metrics-daemon-8d2pt","openshift-network-diagnostics/network-check-target-2z62p","openshift-ovn-kubernetes/ovnkube-node-7zbmk","kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal","openshift-dns/node-resolver-2d6ll","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal","openshift-multus/multus-8sjdt","openshift-multus/multus-additional-cni-plugins-cq5sg","openshift-network-operator/iptables-alerter-nc25r"] Jun 05 13:06:28.702924 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.702905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:28.703009 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.702989 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:28.703787 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.703763 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.704992 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.704971 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.705904 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.705889 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.706166 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.706133 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 05 13:06:28.706166 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.706143 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kzs24\"" Jun 05 13:06:28.706303 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.706218 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 05 13:06:28.706884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.706867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.707240 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.707225 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 05 13:06:28.707441 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.707425 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.707502 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.707428 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fzmm7\"" Jun 05 13:06:28.707663 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.707649 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.707978 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.707960 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:28.708067 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.708022 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:28.708200 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.708175 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6wvgk\"" Jun 05 13:06:28.708279 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.708202 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.708410 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.708395 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.708918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.708904 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.709407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.709219 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tjh25\"" Jun 05 13:06:28.709407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.709284 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.709407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.709317 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.709407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.709304 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 05 13:06:28.709684 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.709653 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 05 13:06:28.710815 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.710799 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.711761 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.711746 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.711856 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.711811 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 05 13:06:28.711856 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.711750 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 05 13:06:28.711984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.711969 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 05 13:06:28.712064 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.712045 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zxj8g\"" Jun 05 13:06:28.712187 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.712171 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.712306 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.712284 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 05 13:06:28.712472 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.712456 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.712904 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.712889 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.712995 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.712963 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 05 13:06:28.713186 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713172 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.713267 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713238 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 05 13:06:28.713267 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713261 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qm9mx\"" Jun 05 13:06:28.713430 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713417 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.713485 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713469 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 05 13:06:28.713983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713966 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7368b0d-6671-4483-8df0-4e42154b7dcd-tmp\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.714081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.713989 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-slash\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714004 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-system-cni-dir\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.714081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714018 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d39febf5-14be-40b7-9861-bea08e0c2405-agent-certs\") pod \"konnectivity-agent-5r9xd\" (UID: \"d39febf5-14be-40b7-9861-bea08e0c2405\") " pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.714081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714034 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-device-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.714081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-os-release\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.714281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714092 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-systemd\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.714281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714147 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-kubelet\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714175 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714198 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysctl-conf\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.714281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714220 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-systemd\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714264 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovnkube-script-lib\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714293 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-t77w7\"" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714311 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-cnibin\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714388 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714430 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-sys\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714458 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-cni-bin\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714482 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-log-socket\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-cni-netd\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-cni-binary-copy\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714562 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 05 13:06:28.714582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714569 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714615 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kkh8\" (UniqueName: \"kubernetes.io/projected/f548aa50-b7b1-473b-8043-e6549fa1f4cc-kube-api-access-9kkh8\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714679 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-kubernetes\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714701 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-var-lib-kubelet\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714725 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2r9\" (UniqueName: \"kubernetes.io/projected/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-kube-api-access-6g2r9\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-host\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714792 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-var-lib-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714826 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-node-log\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714850 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714882 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714918 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9057f24-b694-4da1-be02-c25a9f1e1ef0-host-slash\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714930 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714945 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovnkube-config\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.715005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.714987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ndhq\" (UniqueName: \"kubernetes.io/projected/7f69faaa-f860-47ee-b714-9eb85949cdd5-kube-api-access-8ndhq\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715016 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715041 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-run-netns\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715092 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715437 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d39febf5-14be-40b7-9861-bea08e0c2405-konnectivity-ca\") pod \"konnectivity-agent-5r9xd\" (UID: \"d39febf5-14be-40b7-9861-bea08e0c2405\") " pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715475 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-sys-fs\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715509 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wn4w\" (UniqueName: \"kubernetes.io/projected/d9057f24-b694-4da1-be02-c25a9f1e1ef0-kube-api-access-9wn4w\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-etc-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-socket-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715587 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g5mcn\"" Jun 05 13:06:28.715609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9057f24-b694-4da1-be02-c25a9f1e1ef0-iptables-alerter-script\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715636 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovn-node-metrics-cert\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:28.715678 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f75fd8523075e78d67ffee29b41dde.slice/crio-fdc97be49e35612f71b5825242e54d2a89c126572c886e41cd0cb326c863e8bf WatchSource:0}: Error finding container fdc97be49e35612f71b5825242e54d2a89c126572c886e41cd0cb326c863e8bf: Status 404 returned error can't find the container with id fdc97be49e35612f71b5825242e54d2a89c126572c886e41cd0cb326c863e8bf Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715698 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715751 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-modprobe-d\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715772 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysctl-d\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715818 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715856 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-run\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-tuned\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.715971 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-systemd-units\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716007 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-ovn\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.716085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716030 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9zch\" (UniqueName: \"kubernetes.io/projected/d7368b0d-6671-4483-8df0-4e42154b7dcd-kube-api-access-h9zch\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:28.716155 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5808d78f0aee0231302b30ce2d3d649.slice/crio-7891307731fa0a68a0edd792a3ec770e78eb05a16b3d55616ef8aedf9e5fee79 WatchSource:0}: Error finding container 7891307731fa0a68a0edd792a3ec770e78eb05a16b3d55616ef8aedf9e5fee79: Status 404 returned error can't find the container with id 7891307731fa0a68a0edd792a3ec770e78eb05a16b3d55616ef8aedf9e5fee79 Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716203 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-env-overrides\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716226 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhfp4\" (UniqueName: \"kubernetes.io/projected/f1550b36-46a1-440a-9829-ff5dcec16135-kube-api-access-mhfp4\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716360 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-registration-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716382 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysconfig\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716407 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-lib-modules\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.716546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.716515 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.716808 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.716740 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:28.717229 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.717210 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 05 13:06:28.717326 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.717288 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t7gsw\"" Jun 05 13:06:28.720935 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.720920 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 05 13:06:28.721105 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.721093 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 05 13:06:28.748380 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.748298 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-04 13:01:27 +0000 UTC" deadline="2027-11-28 13:26:12.438893588 +0000 UTC" Jun 05 13:06:28.748380 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.748357 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12984h19m43.690540885s" Jun 05 13:06:28.750623 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.750605 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qr2q7" Jun 05 13:06:28.759134 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.759117 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qr2q7" Jun 05 13:06:28.779918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.779887 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" event={"ID":"24f75fd8523075e78d67ffee29b41dde","Type":"ContainerStarted","Data":"fdc97be49e35612f71b5825242e54d2a89c126572c886e41cd0cb326c863e8bf"} Jun 05 13:06:28.780905 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.780884 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" event={"ID":"c5808d78f0aee0231302b30ce2d3d649","Type":"ContainerStarted","Data":"7891307731fa0a68a0edd792a3ec770e78eb05a16b3d55616ef8aedf9e5fee79"} Jun 05 13:06:28.811165 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.811146 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 05 13:06:28.816876 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-var-lib-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817008 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-node-log\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817008 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816897 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:28.817008 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-k8s-cni-cncf-io\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817008 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816963 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-node-log\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817008 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816962 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-netns\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817008 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816987 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-var-lib-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.816993 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-hostroot\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817044 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rskfb\" (UniqueName: \"kubernetes.io/projected/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-kube-api-access-rskfb\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-os-release\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817126 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-multus-certs\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817146 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817147 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a2c2470-f752-457d-b759-fad45632ee5f-host\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817175 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9057f24-b694-4da1-be02-c25a9f1e1ef0-host-slash\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovnkube-config\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ndhq\" (UniqueName: \"kubernetes.io/projected/7f69faaa-f860-47ee-b714-9eb85949cdd5-kube-api-access-8ndhq\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817286 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9057f24-b694-4da1-be02-c25a9f1e1ef0-host-slash\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:28.817307 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-run-netns\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817371 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-daemon-config\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817380 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-run-netns\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.817399 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817408 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nbgf\" (UniqueName: \"kubernetes.io/projected/d5a7d168-1131-4bee-95e0-0b8d549f10f2-kube-api-access-5nbgf\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817428 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d39febf5-14be-40b7-9861-bea08e0c2405-konnectivity-ca\") pod \"konnectivity-agent-5r9xd\" (UID: \"d39febf5-14be-40b7-9861-bea08e0c2405\") " pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.817459 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:29.317440244 +0000 UTC m=+1.958764043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817481 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-sys-fs\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wn4w\" (UniqueName: \"kubernetes.io/projected/d9057f24-b694-4da1-be02-c25a9f1e1ef0-kube-api-access-9wn4w\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817512 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817533 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-etc-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817560 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-sys-fs\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817575 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-etc-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817651 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-system-cni-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.817889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-socket-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817690 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-cni-multus\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817732 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/840dd6b5-5015-4128-be30-aa29c74fa1e8-dbus\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9057f24-b694-4da1-be02-c25a9f1e1ef0-iptables-alerter-script\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovn-node-metrics-cert\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-cnibin\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817830 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovnkube-config\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817799 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-socket-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5a7d168-1131-4bee-95e0-0b8d549f10f2-cni-binary-copy\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-etc-kubernetes\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817946 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-modprobe-d\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817979 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d39febf5-14be-40b7-9861-bea08e0c2405-konnectivity-ca\") pod \"konnectivity-agent-5r9xd\" (UID: \"d39febf5-14be-40b7-9861-bea08e0c2405\") " pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.817999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysctl-d\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818020 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-run\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818020 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818041 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-tuned\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.818596 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818043 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818062 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-systemd-units\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818083 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-ovn\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-cni-bin\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818127 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-socket-dir-parent\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818140 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-modprobe-d\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818155 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-conf-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818174 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818183 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-systemd-units\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818184 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a2c2470-f752-457d-b759-fad45632ee5f-serviceca\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818154 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysctl-d\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818234 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-run\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818237 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9zch\" (UniqueName: \"kubernetes.io/projected/d7368b0d-6671-4483-8df0-4e42154b7dcd-kube-api-access-h9zch\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818290 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818296 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9057f24-b694-4da1-be02-c25a9f1e1ef0-iptables-alerter-script\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-env-overrides\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818323 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.819421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818290 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-ovn\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhfp4\" (UniqueName: \"kubernetes.io/projected/f1550b36-46a1-440a-9829-ff5dcec16135-kube-api-access-mhfp4\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-tmp-dir\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-registration-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysconfig\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818465 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-registration-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818473 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysconfig\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-lib-modules\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818552 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7368b0d-6671-4483-8df0-4e42154b7dcd-tmp\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818574 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-slash\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-system-cni-dir\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d39febf5-14be-40b7-9861-bea08e0c2405-agent-certs\") pod \"konnectivity-agent-5r9xd\" (UID: \"d39febf5-14be-40b7-9861-bea08e0c2405\") " pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818646 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-env-overrides\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818661 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-lib-modules\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818664 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-slash\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818684 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-system-cni-dir\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818689 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-device-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.820188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818724 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818727 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f548aa50-b7b1-473b-8043-e6549fa1f4cc-device-dir\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-os-release\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818789 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-os-release\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818784 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-openvswitch\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818818 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-kubelet\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818847 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rf4w\" (UniqueName: \"kubernetes.io/projected/3a2c2470-f752-457d-b759-fad45632ee5f-kube-api-access-8rf4w\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-systemd\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818894 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-kubelet\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-cni-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818941 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-systemd\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818959 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysctl-conf\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818962 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-kubelet\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.818989 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819008 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-systemd\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819041 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovnkube-script-lib\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.820665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819095 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-sysctl-conf\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819137 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-run-systemd\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-cnibin\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819175 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819180 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1550b36-46a1-440a-9829-ff5dcec16135-cnibin\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/840dd6b5-5015-4128-be30-aa29c74fa1e8-kubelet-config\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-sys\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819271 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-cni-bin\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819292 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-hosts-file\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819316 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819340 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-log-socket\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819361 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-cni-netd\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-cni-binary-copy\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819410 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kkh8\" (UniqueName: \"kubernetes.io/projected/f548aa50-b7b1-473b-8043-e6549fa1f4cc-kube-api-access-9kkh8\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-kubernetes\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-var-lib-kubelet\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2r9\" (UniqueName: \"kubernetes.io/projected/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-kube-api-access-6g2r9\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:28.821231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-host\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819623 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-host\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819684 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-sys\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819696 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovnkube-script-lib\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819721 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819723 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-cni-bin\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819819 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-log-socket\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819863 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f69faaa-f860-47ee-b714-9eb85949cdd5-host-cni-netd\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819912 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-var-lib-kubelet\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.819953 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-kubernetes\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.820116 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1550b36-46a1-440a-9829-ff5dcec16135-cni-binary-copy\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.821207 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7368b0d-6671-4483-8df0-4e42154b7dcd-tmp\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.821290 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d7368b0d-6671-4483-8df0-4e42154b7dcd-etc-tuned\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.821324 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f69faaa-f860-47ee-b714-9eb85949cdd5-ovn-node-metrics-cert\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.821720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.821442 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d39febf5-14be-40b7-9861-bea08e0c2405-agent-certs\") pod \"konnectivity-agent-5r9xd\" (UID: \"d39febf5-14be-40b7-9861-bea08e0c2405\") " pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:28.827033 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.827014 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:06:28.827033 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.827030 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:06:28.827154 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.827039 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:28.827154 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.827084 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:29.327074185 +0000 UTC m=+1.968397963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:28.828812 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.828786 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wn4w\" (UniqueName: \"kubernetes.io/projected/d9057f24-b694-4da1-be02-c25a9f1e1ef0-kube-api-access-9wn4w\") pod \"iptables-alerter-nc25r\" (UID: \"d9057f24-b694-4da1-be02-c25a9f1e1ef0\") " pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:28.829454 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.829427 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ndhq\" (UniqueName: \"kubernetes.io/projected/7f69faaa-f860-47ee-b714-9eb85949cdd5-kube-api-access-8ndhq\") pod \"ovnkube-node-7zbmk\" (UID: \"7f69faaa-f860-47ee-b714-9eb85949cdd5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:28.829722 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.829703 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2r9\" (UniqueName: \"kubernetes.io/projected/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-kube-api-access-6g2r9\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:28.830058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.830043 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhfp4\" (UniqueName: \"kubernetes.io/projected/f1550b36-46a1-440a-9829-ff5dcec16135-kube-api-access-mhfp4\") pod \"multus-additional-cni-plugins-cq5sg\" (UID: \"f1550b36-46a1-440a-9829-ff5dcec16135\") " pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:28.830577 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.830555 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9zch\" (UniqueName: \"kubernetes.io/projected/d7368b0d-6671-4483-8df0-4e42154b7dcd-kube-api-access-h9zch\") pod \"tuned-fthpr\" (UID: \"d7368b0d-6671-4483-8df0-4e42154b7dcd\") " pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:28.830649 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.830581 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kkh8\" (UniqueName: \"kubernetes.io/projected/f548aa50-b7b1-473b-8043-e6549fa1f4cc-kube-api-access-9kkh8\") pod \"aws-ebs-csi-driver-node-t9b59\" (UID: \"f548aa50-b7b1-473b-8043-e6549fa1f4cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:28.920057 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920033 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-daemon-config\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920150 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nbgf\" (UniqueName: \"kubernetes.io/projected/d5a7d168-1131-4bee-95e0-0b8d549f10f2-kube-api-access-5nbgf\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920150 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-system-cni-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920261 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920151 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-cni-multus\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920261 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920206 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/840dd6b5-5015-4128-be30-aa29c74fa1e8-dbus\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.920261 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-cnibin\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920392 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920374 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5a7d168-1131-4bee-95e0-0b8d549f10f2-cni-binary-copy\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920438 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-etc-kubernetes\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920483 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920436 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-cni-bin\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920483 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920465 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-socket-dir-parent\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-conf-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a2c2470-f752-457d-b759-fad45632ee5f-serviceca\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.920571 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920546 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-tmp-dir\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.920694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920583 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-kubelet\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920614 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rf4w\" (UniqueName: \"kubernetes.io/projected/3a2c2470-f752-457d-b759-fad45632ee5f-kube-api-access-8rf4w\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.920694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920639 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-cni-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920672 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/840dd6b5-5015-4128-be30-aa29c74fa1e8-kubelet-config\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.920868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-hosts-file\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.920868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920729 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.920868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920792 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-k8s-cni-cncf-io\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920821 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-netns\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.920868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-hostroot\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.921079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rskfb\" (UniqueName: \"kubernetes.io/projected/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-kube-api-access-rskfb\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.921079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920905 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-os-release\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.921079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-multus-certs\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.921079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.920961 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a2c2470-f752-457d-b759-fad45632ee5f-host\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.921079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.921068 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a2c2470-f752-457d-b759-fad45632ee5f-host\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.921597 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.921574 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-daemon-config\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.921929 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.921908 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-netns\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.921929 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.921915 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-hostroot\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922056 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.921954 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/840dd6b5-5015-4128-be30-aa29c74fa1e8-kubelet-config\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.922056 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.921987 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-kubelet\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922061 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-hosts-file\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.922141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922073 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-cni-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-system-cni-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922300 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922196 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/840dd6b5-5015-4128-be30-aa29c74fa1e8-dbus\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:28.922300 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922200 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-tmp-dir\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.922300 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.922214 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:28.922300 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:28.922280 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:29.42226632 +0000 UTC m=+2.063590098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:28.922486 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922288 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-cnibin\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922486 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922391 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-etc-kubernetes\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922486 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922460 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-cni-bin\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922820 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922802 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-multus-certs\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922866 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922821 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-socket-dir-parent\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922902 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922891 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-multus-conf-dir\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.922966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.922945 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-run-k8s-cni-cncf-io\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.923095 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.923074 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5a7d168-1131-4bee-95e0-0b8d549f10f2-cni-binary-copy\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.923165 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.923078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-host-var-lib-cni-multus\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.923165 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.923128 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5a7d168-1131-4bee-95e0-0b8d549f10f2-os-release\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.923710 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.923695 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a2c2470-f752-457d-b759-fad45632ee5f-serviceca\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:28.936454 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.936436 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rskfb\" (UniqueName: \"kubernetes.io/projected/e3e1da7e-c5f3-4a04-95c8-230051ddcafe-kube-api-access-rskfb\") pod \"node-resolver-2d6ll\" (UID: \"e3e1da7e-c5f3-4a04-95c8-230051ddcafe\") " pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:28.937117 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.937092 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nbgf\" (UniqueName: \"kubernetes.io/projected/d5a7d168-1131-4bee-95e0-0b8d549f10f2-kube-api-access-5nbgf\") pod \"multus-8sjdt\" (UID: \"d5a7d168-1131-4bee-95e0-0b8d549f10f2\") " pod="openshift-multus/multus-8sjdt" Jun 05 13:06:28.937873 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:28.937853 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rf4w\" (UniqueName: \"kubernetes.io/projected/3a2c2470-f752-457d-b759-fad45632ee5f-kube-api-access-8rf4w\") pod \"node-ca-zg2xk\" (UID: \"3a2c2470-f752-457d-b759-fad45632ee5f\") " pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:29.021980 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.021930 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:29.028193 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.028177 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" Jun 05 13:06:29.030010 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.029960 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39febf5_14be_40b7_9861_bea08e0c2405.slice/crio-48028d517e814151c0ea26e1b820db21a3f3a372ff5e7f04ea8ee0f4828593a3 WatchSource:0}: Error finding container 48028d517e814151c0ea26e1b820db21a3f3a372ff5e7f04ea8ee0f4828593a3: Status 404 returned error can't find the container with id 48028d517e814151c0ea26e1b820db21a3f3a372ff5e7f04ea8ee0f4828593a3 Jun 05 13:06:29.035559 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.035541 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fthpr" Jun 05 13:06:29.035781 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.035761 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf548aa50_b7b1_473b_8043_e6549fa1f4cc.slice/crio-aa3e3b577cd0cb7a0ffc955182187af9330fb5323820192736297d3536a20e81 WatchSource:0}: Error finding container aa3e3b577cd0cb7a0ffc955182187af9330fb5323820192736297d3536a20e81: Status 404 returned error can't find the container with id aa3e3b577cd0cb7a0ffc955182187af9330fb5323820192736297d3536a20e81 Jun 05 13:06:29.041780 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.041678 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nc25r" Jun 05 13:06:29.042639 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.042616 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7368b0d_6671_4483_8df0_4e42154b7dcd.slice/crio-7bdf1816555de7b50908915b203a31ef0f948285d6c4de3ca40a70ef62305048 WatchSource:0}: Error finding container 7bdf1816555de7b50908915b203a31ef0f948285d6c4de3ca40a70ef62305048: Status 404 returned error can't find the container with id 7bdf1816555de7b50908915b203a31ef0f948285d6c4de3ca40a70ef62305048 Jun 05 13:06:29.046049 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.046034 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:29.049205 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.049188 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9057f24_b694_4da1_be02_c25a9f1e1ef0.slice/crio-fe603226b6a64954847ba8ea8be6b8e81aee39a061fdaa035cb77161452319e3 WatchSource:0}: Error finding container fe603226b6a64954847ba8ea8be6b8e81aee39a061fdaa035cb77161452319e3: Status 404 returned error can't find the container with id fe603226b6a64954847ba8ea8be6b8e81aee39a061fdaa035cb77161452319e3 Jun 05 13:06:29.052402 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.052388 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" Jun 05 13:06:29.053374 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.053357 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f69faaa_f860_47ee_b714_9eb85949cdd5.slice/crio-7fcee3fc03659de813dc7afbd8916ef0e0b1ac22b5c1c045a310941ce6057728 WatchSource:0}: Error finding container 7fcee3fc03659de813dc7afbd8916ef0e0b1ac22b5c1c045a310941ce6057728: Status 404 returned error can't find the container with id 7fcee3fc03659de813dc7afbd8916ef0e0b1ac22b5c1c045a310941ce6057728 Jun 05 13:06:29.056238 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.056223 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zg2xk" Jun 05 13:06:29.059353 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.059332 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1550b36_46a1_440a_9829_ff5dcec16135.slice/crio-3b623d97d3f0ba5c9e68bf4084c748995167eaf2c85b4ffdf14c8410ff9f6456 WatchSource:0}: Error finding container 3b623d97d3f0ba5c9e68bf4084c748995167eaf2c85b4ffdf14c8410ff9f6456: Status 404 returned error can't find the container with id 3b623d97d3f0ba5c9e68bf4084c748995167eaf2c85b4ffdf14c8410ff9f6456 Jun 05 13:06:29.061197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.061180 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2d6ll" Jun 05 13:06:29.062763 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.062743 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2c2470_f752_457d_b759_fad45632ee5f.slice/crio-fb94f4db686648e485a608b94acf2d661119cab19d80138675a2616788b9602a WatchSource:0}: Error finding container fb94f4db686648e485a608b94acf2d661119cab19d80138675a2616788b9602a: Status 404 returned error can't find the container with id fb94f4db686648e485a608b94acf2d661119cab19d80138675a2616788b9602a Jun 05 13:06:29.065455 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.065437 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8sjdt" Jun 05 13:06:29.068677 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.068657 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e1da7e_c5f3_4a04_95c8_230051ddcafe.slice/crio-de6d806c45166c12db99196ac31f22cdb8cfcd2a5c31eeb43edbbd2862c2a620 WatchSource:0}: Error finding container de6d806c45166c12db99196ac31f22cdb8cfcd2a5c31eeb43edbbd2862c2a620: Status 404 returned error can't find the container with id de6d806c45166c12db99196ac31f22cdb8cfcd2a5c31eeb43edbbd2862c2a620 Jun 05 13:06:29.072726 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:06:29.072708 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a7d168_1131_4bee_95e0_0b8d549f10f2.slice/crio-79fc6db62ffa3d58512d2e354c028fd4cb7d79d7be3736da4fe2b4ca80d59bbe WatchSource:0}: Error finding container 79fc6db62ffa3d58512d2e354c028fd4cb7d79d7be3736da4fe2b4ca80d59bbe: Status 404 returned error can't find the container with id 79fc6db62ffa3d58512d2e354c028fd4cb7d79d7be3736da4fe2b4ca80d59bbe Jun 05 13:06:29.323219 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.323149 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:29.323389 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.323281 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:29.323389 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.323337 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:30.323321156 +0000 UTC m=+2.964644939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:29.423423 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.423394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:29.423583 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.423441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:29.423583 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.423548 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:29.423698 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.423601 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:06:29.423698 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.423620 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:30.423599994 +0000 UTC m=+3.064923776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:29.423698 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.423625 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:06:29.423698 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.423638 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:29.423698 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:29.423690 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:30.423674391 +0000 UTC m=+3.064998184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:29.761637 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.761381 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-04 13:01:28 +0000 UTC" deadline="2028-01-31 19:56:58.766335485 +0000 UTC" Jun 05 13:06:29.761637 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.761638 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14526h50m29.004703982s" Jun 05 13:06:29.795050 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.795016 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8sjdt" event={"ID":"d5a7d168-1131-4bee-95e0-0b8d549f10f2","Type":"ContainerStarted","Data":"79fc6db62ffa3d58512d2e354c028fd4cb7d79d7be3736da4fe2b4ca80d59bbe"} Jun 05 13:06:29.802969 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.802940 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2d6ll" event={"ID":"e3e1da7e-c5f3-4a04-95c8-230051ddcafe","Type":"ContainerStarted","Data":"de6d806c45166c12db99196ac31f22cdb8cfcd2a5c31eeb43edbbd2862c2a620"} Jun 05 13:06:29.811414 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.811380 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"7fcee3fc03659de813dc7afbd8916ef0e0b1ac22b5c1c045a310941ce6057728"} Jun 05 13:06:29.827767 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.827742 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zg2xk" event={"ID":"3a2c2470-f752-457d-b759-fad45632ee5f","Type":"ContainerStarted","Data":"fb94f4db686648e485a608b94acf2d661119cab19d80138675a2616788b9602a"} Jun 05 13:06:29.836371 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.836352 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:29.839182 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.839129 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerStarted","Data":"3b623d97d3f0ba5c9e68bf4084c748995167eaf2c85b4ffdf14c8410ff9f6456"} Jun 05 13:06:29.845827 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.845804 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nc25r" event={"ID":"d9057f24-b694-4da1-be02-c25a9f1e1ef0","Type":"ContainerStarted","Data":"fe603226b6a64954847ba8ea8be6b8e81aee39a061fdaa035cb77161452319e3"} Jun 05 13:06:29.865304 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.865279 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fthpr" event={"ID":"d7368b0d-6671-4483-8df0-4e42154b7dcd","Type":"ContainerStarted","Data":"7bdf1816555de7b50908915b203a31ef0f948285d6c4de3ca40a70ef62305048"} Jun 05 13:06:29.880375 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.880275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" event={"ID":"f548aa50-b7b1-473b-8043-e6549fa1f4cc","Type":"ContainerStarted","Data":"aa3e3b577cd0cb7a0ffc955182187af9330fb5323820192736297d3536a20e81"} Jun 05 13:06:29.888950 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:29.888927 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5r9xd" event={"ID":"d39febf5-14be-40b7-9861-bea08e0c2405","Type":"ContainerStarted","Data":"48028d517e814151c0ea26e1b820db21a3f3a372ff5e7f04ea8ee0f4828593a3"} Jun 05 13:06:30.331921 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.331844 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:30.332076 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.332004 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:30.332137 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.332079 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:32.332060073 +0000 UTC m=+4.973383854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:30.433038 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.432997 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:30.433204 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.433045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:30.433284 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.433207 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:06:30.433284 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.433225 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:06:30.433284 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.433237 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:30.433430 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.433308 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:32.433289787 +0000 UTC m=+5.074613569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:30.433709 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.433688 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:30.433786 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.433743 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:32.433729495 +0000 UTC m=+5.075053279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:30.762171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.762065 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-04 13:01:28 +0000 UTC" deadline="2028-03-16 01:48:26.528222244 +0000 UTC" Jun 05 13:06:30.762171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.762103 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15588h41m55.766123763s" Jun 05 13:06:30.777517 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.777423 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:30.777698 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.777581 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:30.778070 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.777787 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:30.778070 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.777914 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:30.778070 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:30.777428 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:30.778070 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:30.778012 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:31.035123 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:31.034987 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:31.779710 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:31.779120 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 05 13:06:32.351281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:32.351226 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:32.351436 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.351373 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:32.351503 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.351455 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:36.351434001 +0000 UTC m=+8.992757793 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:32.451814 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:32.451759 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:32.451814 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:32.451817 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:32.452043 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.451974 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:06:32.452043 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.451992 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:06:32.452043 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.452004 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:32.452184 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.452060 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:36.452043306 +0000 UTC m=+9.093367087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:32.452591 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.452464 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:32.452591 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.452517 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:36.452503325 +0000 UTC m=+9.093827106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:32.777395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:32.777310 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:32.777395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:32.777342 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:32.777395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:32.777357 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:32.777652 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.777444 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:32.777652 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.777513 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:32.777652 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:32.777594 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:34.776762 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:34.776724 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:34.777164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:34.776851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:34.777164 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:34.776863 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:34.777164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:34.776879 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:34.777164 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:34.776963 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:34.777164 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:34.777048 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:36.383196 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:36.383152 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:36.383646 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.383336 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:36.383646 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.383418 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:44.3833978 +0000 UTC m=+17.024721589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:36.483604 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:36.483563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:36.483773 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:36.483619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:36.483846 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.483792 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:06:36.483846 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.483810 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:06:36.483846 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.483822 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:36.483980 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.483882 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:36.483980 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.483888 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:44.483870234 +0000 UTC m=+17.125194015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:36.483980 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.483959 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:06:44.483941552 +0000 UTC m=+17.125265333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:36.776653 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:36.776571 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:36.776653 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:36.776601 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:36.776653 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:36.776615 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:36.776957 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.776706 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:36.776957 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.776906 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:36.777069 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:36.777014 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:38.776618 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:38.776585 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:38.776618 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:38.776611 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:38.776998 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:38.776625 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:38.776998 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:38.776728 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:38.776998 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:38.776821 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:38.776998 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:38.776900 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:40.776783 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:40.776758 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:40.777177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:40.776871 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:40.777177 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:40.777000 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:40.777177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:40.776877 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:40.777177 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:40.776873 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:40.777177 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:40.777113 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:42.776787 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:42.776755 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:42.777283 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:42.776755 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:42.777283 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:42.776882 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:42.777283 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:42.776756 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:42.777283 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:42.776932 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:42.777283 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:42.777034 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:44.443058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:44.443015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:44.443567 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.443151 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:44.443567 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.443223 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:07:00.443199801 +0000 UTC m=+33.084523594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:06:44.544197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:44.544159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:44.544397 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:44.544208 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:44.544397 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.544351 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:06:44.544397 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.544370 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:06:44.544397 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.544371 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:44.544588 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.544443 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:07:00.544423272 +0000 UTC m=+33.185747066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:06:44.544588 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.544380 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:44.544588 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.544538 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:07:00.544523581 +0000 UTC m=+33.185847372 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:06:44.777144 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:44.777111 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:44.777144 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:44.777146 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:44.777366 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:44.777242 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:44.777366 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.777268 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:44.777366 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.777336 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:44.777522 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:44.777401 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:46.777209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:46.777169 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:46.777655 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:46.777188 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:46.777655 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:46.777320 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:46.777655 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:46.777375 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:46.777655 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:46.777494 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:46.777655 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:46.777620 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:48.776529 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.776222 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:48.776884 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:48.776581 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:48.776884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.776421 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:48.776884 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:48.776706 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:48.776884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.776383 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:48.776884 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:48.776771 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:48.941240 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.941019 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8sjdt" event={"ID":"d5a7d168-1131-4bee-95e0-0b8d549f10f2","Type":"ContainerStarted","Data":"504b72a232ad4515f8e5146ef6b505a29aa6b3f5d074008853dc9c08425c9607"} Jun 05 13:06:48.942541 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.942517 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"4288103cf960083307d9afd0990e19008e266de6541e84257c5385287bb0179a"} Jun 05 13:06:48.944069 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.943877 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fthpr" event={"ID":"d7368b0d-6671-4483-8df0-4e42154b7dcd","Type":"ContainerStarted","Data":"7ef04d7a5ea5c07cbd65c05d3367e240fcf90de61355740db0191cafb867d809"} Jun 05 13:06:48.945573 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.945554 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" event={"ID":"24f75fd8523075e78d67ffee29b41dde","Type":"ContainerStarted","Data":"fe1e75ba61f27ff0fc164bfc1312d4918a4a107f7acf31e78e2b4f4f489141ba"} Jun 05 13:06:48.973634 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.973593 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-104.ec2.internal" podStartSLOduration=20.973574911 podStartE2EDuration="20.973574911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:48.973311374 +0000 UTC m=+21.614635174" watchObservedRunningTime="2026-06-05 13:06:48.973574911 +0000 UTC m=+21.614898711" Jun 05 13:06:48.973750 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.973727 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8sjdt" podStartSLOduration=1.908245622 podStartE2EDuration="20.973720335s" totalImagesPullingTime="19.065474713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:48.958508629 +0000 UTC m=+21.599832429" watchObservedRunningTime="2026-06-05 13:06:48.973720335 +0000 UTC m=+21.615044134" Jun 05 13:06:48.992959 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:48.992922 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fthpr" podStartSLOduration=2.931191874 podStartE2EDuration="21.992908327s" totalImagesPullingTime="19.061716453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:48.992773276 +0000 UTC m=+21.634097075" watchObservedRunningTime="2026-06-05 13:06:48.992908327 +0000 UTC m=+21.634232126" Jun 05 13:06:49.948902 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.948871 2570 generic.go:356] "Generic (PLEG): container finished" podID="c5808d78f0aee0231302b30ce2d3d649" containerID="aa859ce81c595ba2905cc731f05e90a0e4e1c161ec9b929e2aeb0dc502b8370e" exitCode=0 Jun 05 13:06:49.949650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.948949 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" event={"ID":"c5808d78f0aee0231302b30ce2d3d649","Type":"ContainerDied","Data":"aa859ce81c595ba2905cc731f05e90a0e4e1c161ec9b929e2aeb0dc502b8370e"} Jun 05 13:06:49.950465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.950437 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2d6ll" event={"ID":"e3e1da7e-c5f3-4a04-95c8-230051ddcafe","Type":"ContainerStarted","Data":"2246f2932c5c7f91b7e7d3787520155fd317045df6d73d2f785aa4242d6dd19c"} Jun 05 13:06:49.953430 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.953400 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"7d01816bcce5dbe5bfdf60271fe4577ad6b172d279675aef61e3123e3225d1c4"} Jun 05 13:06:49.953516 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.953458 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"caa2315fd96f9728551bb4fb22533e226b90fc894dadac95b8cd6d0be07cff1b"} Jun 05 13:06:49.953516 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.953487 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"8f43240f5a1de8a37be5faf2a72c370933bcf8f1ca7862fe52545c998185f9dd"} Jun 05 13:06:49.953516 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.953513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"aced2c0b1339369f2bceef371045d8854536d82b0e8d487aac3f417eb3514f36"} Jun 05 13:06:49.953627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.953540 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"e4bbd97d1e8ac9631a37c703b9f50e5f03913579ddff558e01e833b610d25c82"} Jun 05 13:06:49.954618 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.954599 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zg2xk" event={"ID":"3a2c2470-f752-457d-b759-fad45632ee5f","Type":"ContainerStarted","Data":"7a6b84717975919b344aaa488be448f2851f5a91afd0f218cce54bbab649c82e"} Jun 05 13:06:49.955906 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.955884 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1550b36-46a1-440a-9829-ff5dcec16135" containerID="6a02c6aa8c560172d9ae4918cb0e6b4f463a77d4eb8d5d0dbc0c42ae374408ea" exitCode=0 Jun 05 13:06:49.955981 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.955957 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerDied","Data":"6a02c6aa8c560172d9ae4918cb0e6b4f463a77d4eb8d5d0dbc0c42ae374408ea"} Jun 05 13:06:49.957300 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.957274 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nc25r" event={"ID":"d9057f24-b694-4da1-be02-c25a9f1e1ef0","Type":"ContainerStarted","Data":"be18b7e1dd3f696da7db45d3ba343bb72f84804de41c93d21b62e8f7b7031b9c"} Jun 05 13:06:49.958705 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.958683 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" event={"ID":"f548aa50-b7b1-473b-8043-e6549fa1f4cc","Type":"ContainerStarted","Data":"8d368aa7dac0fd8891a6f0a74e6aadc554775c98d576c81a5edc50ef291d6680"} Jun 05 13:06:49.959975 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.959953 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5r9xd" event={"ID":"d39febf5-14be-40b7-9861-bea08e0c2405","Type":"ContainerStarted","Data":"0a4f3f0ca18b789d966082a32c62ac7ecb041c854c7c155910a1226547b12371"} Jun 05 13:06:49.981603 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.981571 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5r9xd" podStartSLOduration=4.247746924 podStartE2EDuration="22.981559848s" totalImagesPullingTime="18.733812924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:49.981186536 +0000 UTC m=+22.622510367" watchObservedRunningTime="2026-06-05 13:06:49.981559848 +0000 UTC m=+22.622883647" Jun 05 13:06:49.995770 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:49.995736 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nc25r" podStartSLOduration=3.940717007 podStartE2EDuration="22.995725026s" totalImagesPullingTime="19.055008019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:49.995644293 +0000 UTC m=+22.636968092" watchObservedRunningTime="2026-06-05 13:06:49.995725026 +0000 UTC m=+22.637048826" Jun 05 13:06:50.033966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.033929 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zg2xk" podStartSLOduration=3.333843346 podStartE2EDuration="22.033917707s" totalImagesPullingTime="18.700074361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:50.033546608 +0000 UTC m=+22.674870418" watchObservedRunningTime="2026-06-05 13:06:50.033917707 +0000 UTC m=+22.675241506" Jun 05 13:06:50.048449 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.048415 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2d6ll" podStartSLOduration=3.353179389 podStartE2EDuration="22.048404671s" totalImagesPullingTime="18.695225282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:50.048320767 +0000 UTC m=+22.689644567" watchObservedRunningTime="2026-06-05 13:06:50.048404671 +0000 UTC m=+22.689728470" Jun 05 13:06:50.200591 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.200524 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 05 13:06:50.776914 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.776883 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:50.777179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.776978 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:50.777179 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:50.776991 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:50.777179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.777002 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:50.777179 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:50.777055 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:50.777363 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:50.777164 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:50.778946 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.778872 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-05T13:06:50.200539807Z","UUID":"6fd58a91-85fa-42d6-8830-f7609d23c253","Handler":null,"Name":"","Endpoint":""} Jun 05 13:06:50.780694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.780676 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 05 13:06:50.780821 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.780717 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 05 13:06:50.964324 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.964285 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" event={"ID":"f548aa50-b7b1-473b-8043-e6549fa1f4cc","Type":"ContainerStarted","Data":"53aff0b0e891363e7a3bb0892bcac37819092ffeca68b324532ac2cbc9d9a72d"} Jun 05 13:06:50.967388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:50.967290 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" event={"ID":"c5808d78f0aee0231302b30ce2d3d649","Type":"ContainerStarted","Data":"50374142d8037ced9d7d46e3ebe43522ee1486e9bab0e647b4fbb9c92e980421"} Jun 05 13:06:51.805426 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.805193 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:51.805825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.805804 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:51.821700 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.821655 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-104.ec2.internal" podStartSLOduration=23.821642364 podStartE2EDuration="23.821642364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:50.983508696 +0000 UTC m=+23.624832495" watchObservedRunningTime="2026-06-05 13:06:51.821642364 +0000 UTC m=+24.462966164" Jun 05 13:06:51.972994 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.972960 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"782a98c154a59c959e2917b10dc304dd7198dc5dcfa315d5cdb248f989751ce1"} Jun 05 13:06:51.975206 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.975179 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" event={"ID":"f548aa50-b7b1-473b-8043-e6549fa1f4cc","Type":"ContainerStarted","Data":"72f465aba79176f48834edf4e09f9f0b9e67fe178e2e0e1f11eed1a8bcb27c3d"} Jun 05 13:06:51.990832 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.990810 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:51.991437 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.991416 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5r9xd" Jun 05 13:06:51.994118 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:51.994081 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t9b59" podStartSLOduration=3.700818745 podStartE2EDuration="24.994064402s" totalImagesPullingTime="21.293245657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:51.99370837 +0000 UTC m=+24.635032171" watchObservedRunningTime="2026-06-05 13:06:51.994064402 +0000 UTC m=+24.635388203" Jun 05 13:06:52.777032 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:52.776997 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:52.777032 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:52.777030 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:52.777277 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:52.777040 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:52.777277 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:52.777123 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:52.777277 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:52.777224 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:52.777404 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:52.777317 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:54.777015 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.776844 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:54.777637 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.776850 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:54.777637 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:54.777077 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:54.777637 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.776850 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:54.777637 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:54.777156 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:54.777637 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:54.777224 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:54.983024 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.982992 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1550b36-46a1-440a-9829-ff5dcec16135" containerID="a3913e779183b1207bb67e968a11877c57c3c135a3ee8c03db7718c6bb0ae9ef" exitCode=0 Jun 05 13:06:54.983213 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.983031 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerDied","Data":"a3913e779183b1207bb67e968a11877c57c3c135a3ee8c03db7718c6bb0ae9ef"} Jun 05 13:06:54.986611 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.986589 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" event={"ID":"7f69faaa-f860-47ee-b714-9eb85949cdd5","Type":"ContainerStarted","Data":"c7975598ceee8b42e5e03171b19a6463a9cd490b81c6a298ef7f363dca9a11d7"} Jun 05 13:06:54.986858 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.986834 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:54.986966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.986892 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:54.986966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:54.986937 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:55.001171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.001151 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:55.001333 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.001317 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:06:55.031434 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.031352 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" podStartSLOduration=8.357332911 podStartE2EDuration="28.03133876s" totalImagesPullingTime="19.674005849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:06:55.031006833 +0000 UTC m=+27.672330652" watchObservedRunningTime="2026-06-05 13:06:55.03133876 +0000 UTC m=+27.672662558" Jun 05 13:06:55.910536 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.910506 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2z62p"] Jun 05 13:06:55.910996 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.910602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:55.910996 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:55.910682 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:55.914439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.914092 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8d2pt"] Jun 05 13:06:55.914439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.914380 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:55.914707 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:55.914489 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:55.914849 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.914825 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q58h"] Jun 05 13:06:55.914974 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:55.914902 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:55.915031 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:55.914992 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:56.992663 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:56.992628 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1550b36-46a1-440a-9829-ff5dcec16135" containerID="de9e45085fddfabe9b30b726c5bde22676c7a37baae2eaf73ecf0ca154bdf735" exitCode=0 Jun 05 13:06:56.993020 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:56.992708 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerDied","Data":"de9e45085fddfabe9b30b726c5bde22676c7a37baae2eaf73ecf0ca154bdf735"} Jun 05 13:06:57.777657 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:57.777583 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:57.777799 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:57.777703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:57.777799 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:57.777724 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:06:57.777799 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:57.777783 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:57.777970 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:57.777829 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:57.777970 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:57.777886 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:58.998528 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:58.998309 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1550b36-46a1-440a-9829-ff5dcec16135" containerID="b81be77c5ec4eeceefdce31329b49b271cd36da594d0440ef89a4b9b07ff7903" exitCode=0 Jun 05 13:06:58.998925 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:58.998384 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerDied","Data":"b81be77c5ec4eeceefdce31329b49b271cd36da594d0440ef89a4b9b07ff7903"} Jun 05 13:06:59.777263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:59.777213 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:06:59.777534 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:59.777353 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2z62p" podUID="8541b5bf-1e1a-4c0a-b395-ef58d894cbe7" Jun 05 13:06:59.777534 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:59.777369 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:06:59.777534 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:06:59.777397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:06:59.777534 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:59.777496 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:06:59.777835 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:06:59.777583 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q58h" podUID="840dd6b5-5015-4128-be30-aa29c74fa1e8" Jun 05 13:07:00.475205 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:00.475176 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:07:00.475815 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.475331 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:07:00.475815 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.475390 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:07:32.475374327 +0000 UTC m=+65.116698114 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 05 13:07:00.576060 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:00.576032 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:07:00.576260 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:00.576077 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:07:00.576260 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.576230 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 05 13:07:00.576381 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.576312 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 05 13:07:00.576381 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.576329 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret podName:840dd6b5-5015-4128-be30-aa29c74fa1e8 nodeName:}" failed. No retries permitted until 2026-06-05 13:07:32.576311369 +0000 UTC m=+65.217635166 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret") pod "global-pull-secret-syncer-7q58h" (UID: "840dd6b5-5015-4128-be30-aa29c74fa1e8") : object "kube-system"/"original-pull-secret" not registered Jun 05 13:07:00.576381 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.576331 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 05 13:07:00.576381 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.576354 2570 projected.go:196] Error preparing data for projected volume kube-api-access-p2nsw for pod openshift-network-diagnostics/network-check-target-2z62p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:07:00.576381 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:00.576380 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw podName:8541b5bf-1e1a-4c0a-b395-ef58d894cbe7 nodeName:}" failed. No retries permitted until 2026-06-05 13:07:32.576373411 +0000 UTC m=+65.217697189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2nsw" (UniqueName: "kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw") pod "network-check-target-2z62p" (UID: "8541b5bf-1e1a-4c0a-b395-ef58d894cbe7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 05 13:07:01.198892 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.198869 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-104.ec2.internal" event="NodeReady" Jun 05 13:07:01.199076 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.199005 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 05 13:07:01.232575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.232539 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f876c6468-qmzf4"] Jun 05 13:07:01.235870 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.235851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.238837 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.238812 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 05 13:07:01.238939 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.238848 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 05 13:07:01.238939 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.238870 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m678x\"" Jun 05 13:07:01.238939 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.238881 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 05 13:07:01.251268 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.248742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 05 13:07:01.253174 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.253150 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f876c6468-qmzf4"] Jun 05 13:07:01.253942 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.253923 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t27mz"] Jun 05 13:07:01.256879 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.256861 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d2nvt"] Jun 05 13:07:01.257011 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.256994 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.259689 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.259634 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.259790 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.259745 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 05 13:07:01.259848 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.259792 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d25hl\"" Jun 05 13:07:01.260164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.260146 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 05 13:07:01.262169 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.262150 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 05 13:07:01.262169 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.262166 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 05 13:07:01.262667 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.262191 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 05 13:07:01.262667 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.262348 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ztm5p\"" Jun 05 13:07:01.266010 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.265979 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t27mz"] Jun 05 13:07:01.270146 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.269883 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d2nvt"] Jun 05 13:07:01.381079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381002 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-config-volume\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.381079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xll\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-kube-api-access-h7xll\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381079 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381058 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-tmp-dir\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381106 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-installation-pull-secrets\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381263 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-bound-sa-token\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99b4q\" (UniqueName: \"kubernetes.io/projected/f5dcc392-3021-4ab6-8b13-a85ec0cfedf4-kube-api-access-99b4q\") pod \"ingress-canary-d2nvt\" (UID: \"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4\") " pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381314 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwfc7\" (UniqueName: \"kubernetes.io/projected/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-kube-api-access-rwfc7\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.381360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381338 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-image-registry-private-configuration\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381656 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-certificates\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381656 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-trusted-ca\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381656 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381451 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c5754e-c750-4fdf-a2e5-19a92fa6671e-ca-trust-extracted\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.381656 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.381475 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5dcc392-3021-4ab6-8b13-a85ec0cfedf4-cert\") pod \"ingress-canary-d2nvt\" (UID: \"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4\") " pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.405652 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.405605 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:01.482622 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482582 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-bound-sa-token\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99b4q\" (UniqueName: \"kubernetes.io/projected/f5dcc392-3021-4ab6-8b13-a85ec0cfedf4-kube-api-access-99b4q\") pod \"ingress-canary-d2nvt\" (UID: \"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4\") " pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwfc7\" (UniqueName: \"kubernetes.io/projected/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-kube-api-access-rwfc7\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.482725 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-image-registry-private-configuration\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-certificates\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.482798 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:01.982778323 +0000 UTC m=+34.624102113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482918 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-trusted-ca\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482972 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c5754e-c750-4fdf-a2e5-19a92fa6671e-ca-trust-extracted\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.482995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5dcc392-3021-4ab6-8b13-a85ec0cfedf4-cert\") pod \"ingress-canary-d2nvt\" (UID: \"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4\") " pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.483039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-config-volume\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xll\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-kube-api-access-h7xll\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483102 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-tmp-dir\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483142 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-installation-pull-secrets\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483426 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-certificates\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.483602 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.483620 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483653 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c5754e-c750-4fdf-a2e5-19a92fa6671e-ca-trust-extracted\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.483680 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:01.983663801 +0000 UTC m=+34.624987582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:01.483734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-tmp-dir\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.484211 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.483859 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-trusted-ca\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.484560 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.484535 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-config-volume\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.487425 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.487405 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-installation-pull-secrets\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.487425 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.487413 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5dcc392-3021-4ab6-8b13-a85ec0cfedf4-cert\") pod \"ingress-canary-d2nvt\" (UID: \"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4\") " pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.487575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.487410 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-image-registry-private-configuration\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.494083 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.494062 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwfc7\" (UniqueName: \"kubernetes.io/projected/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-kube-api-access-rwfc7\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.494620 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.494409 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99b4q\" (UniqueName: \"kubernetes.io/projected/f5dcc392-3021-4ab6-8b13-a85ec0cfedf4-kube-api-access-99b4q\") pod \"ingress-canary-d2nvt\" (UID: \"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4\") " pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.494620 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.494465 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xll\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-kube-api-access-h7xll\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.495007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.494985 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-bound-sa-token\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.500580 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.500522 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:01.573835 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.573801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d2nvt" Jun 05 13:07:01.720078 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.719882 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d2nvt"] Jun 05 13:07:01.724751 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:07:01.724710 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dcc392_3021_4ab6_8b13_a85ec0cfedf4.slice/crio-0a1931b2d04fa688dafd7377e5260409146e51a17950f9fadf41f28cb001353c WatchSource:0}: Error finding container 0a1931b2d04fa688dafd7377e5260409146e51a17950f9fadf41f28cb001353c: Status 404 returned error can't find the container with id 0a1931b2d04fa688dafd7377e5260409146e51a17950f9fadf41f28cb001353c Jun 05 13:07:01.777156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.777124 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:07:01.777311 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.777162 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:07:01.777311 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.777127 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:07:01.779639 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.779617 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 05 13:07:01.779750 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.779672 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nhxzk\"" Jun 05 13:07:01.779750 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.779734 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cph74\"" Jun 05 13:07:01.780033 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.779988 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 05 13:07:01.780033 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.780019 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 05 13:07:01.780315 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.780292 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 05 13:07:01.799621 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.799599 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:01.868354 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.868317 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:01.930215 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.930092 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:01.986894 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.986849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:01.987057 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:01.986908 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:01.987057 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.987003 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:01.987057 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.987012 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:01.987225 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.987098 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:02.987076925 +0000 UTC m=+35.628400709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:01.987225 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.987021 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:01.987225 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:01.987195 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:02.987172722 +0000 UTC m=+35.628496514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:02.007262 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.007219 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d2nvt" event={"ID":"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4","Type":"ContainerStarted","Data":"0a1931b2d04fa688dafd7377e5260409146e51a17950f9fadf41f28cb001353c"} Jun 05 13:07:02.768950 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.768912 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:02.844001 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.843952 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:02.909317 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.909272 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:02.971504 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.971469 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:02.994625 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.994595 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:02.994792 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:02.994646 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:02.994792 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:02.994747 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:02.994792 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:02.994755 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:02.994792 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:02.994772 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:02.994946 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:02.994812 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:04.994800308 +0000 UTC m=+37.636124086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:02.994946 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:02.994827 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:04.994820008 +0000 UTC m=+37.636143786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:04.016257 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:04.016196 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1550b36-46a1-440a-9829-ff5dcec16135" containerID="3f7e6b936c72c200cdfe0c15d12b3625e761614ad1316ab30fe63e282bcb2f73" exitCode=0 Jun 05 13:07:04.016716 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:04.016283 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerDied","Data":"3f7e6b936c72c200cdfe0c15d12b3625e761614ad1316ab30fe63e282bcb2f73"} Jun 05 13:07:05.010156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.010123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:05.010375 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.010177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:05.010375 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:05.010234 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:05.010375 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:05.010269 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:05.010375 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:05.010327 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:09.010311958 +0000 UTC m=+41.651635755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:05.010375 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:05.010338 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:05.010639 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:05.010395 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:09.010377299 +0000 UTC m=+41.651701098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:05.021955 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.021925 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1550b36-46a1-440a-9829-ff5dcec16135" containerID="5f23b20d577cfb5b1d33f41ce9ec3d115eedf27b06757709abb8b420f6044b1b" exitCode=0 Jun 05 13:07:05.022379 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.021976 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerDied","Data":"5f23b20d577cfb5b1d33f41ce9ec3d115eedf27b06757709abb8b420f6044b1b"} Jun 05 13:07:05.557184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.556993 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:05.617447 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.617419 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:05.970601 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:05.970524 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:06.025816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.025790 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d2nvt" event={"ID":"f5dcc392-3021-4ab6-8b13-a85ec0cfedf4","Type":"ContainerStarted","Data":"811ec1b747f7fbcf3fc15a713214200d429d4c36ba103320bec3483cbd0dd761"} Jun 05 13:07:06.029028 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.029004 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" event={"ID":"f1550b36-46a1-440a-9829-ff5dcec16135","Type":"ContainerStarted","Data":"37e20c44ece859128e7edc4aa8d9556b1784b040d073736f2560b4e161630a7d"} Jun 05 13:07:06.031712 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.031686 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:06.041172 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.041129 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d2nvt" podStartSLOduration=1.5172498920000002 podStartE2EDuration="5.041116627s" totalImagesPullingTime="3.523866735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:07:06.04077748 +0000 UTC m=+38.682101279" watchObservedRunningTime="2026-06-05 13:07:06.041116627 +0000 UTC m=+38.682440469" Jun 05 13:07:06.062578 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.062538 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cq5sg" podStartSLOduration=9.06286925 podStartE2EDuration="39.062525813s" totalImagesPullingTime="29.999656563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:07:06.061113399 +0000 UTC m=+38.702437200" watchObservedRunningTime="2026-06-05 13:07:06.062525813 +0000 UTC m=+38.703849613" Jun 05 13:07:06.190610 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.190584 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:06.291465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.291443 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:06.391299 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.391275 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:06.494765 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:06.494740 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:07.804536 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:07.804501 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:08.691605 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:08.691568 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:08.757402 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:08.757373 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:09.041349 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:09.041325 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:09.041678 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:09.041360 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:09.041678 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:09.041450 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:09.041678 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:09.041456 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:09.041678 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:09.041471 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:09.041678 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:09.041495 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:17.041484116 +0000 UTC m=+49.682807895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:09.041678 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:09.041512 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:17.04150132 +0000 UTC m=+49.682825102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:14.048939 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:14.048907 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:16.876705 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:16.876670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:16.975775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:16.975741 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:17.078236 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:17.078208 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:17.092419 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:17.092397 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:17.092551 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:17.092443 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:17.092619 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:17.092550 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:17.092619 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:17.092564 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:17.092619 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:17.092565 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:17.092725 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:17.092625 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:33.092608481 +0000 UTC m=+65.733932276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:17.092725 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:17.092639 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:07:33.092633296 +0000 UTC m=+65.733957074 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:18.955837 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:18.955804 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:19.015959 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:19.015925 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:26.216787 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:26.216755 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:26.317164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:26.317139 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:27.005574 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:27.005548 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zbmk" Jun 05 13:07:27.160818 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:27.160788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:27.222172 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:27.222140 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:32.494030 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.493990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:07:32.496854 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.496830 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 05 13:07:32.504964 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:32.504947 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 05 13:07:32.505027 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:32.505016 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:08:36.50499995 +0000 UTC m=+129.146323728 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : secret "metrics-daemon-secret" not found Jun 05 13:07:32.595233 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.595199 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:07:32.595361 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.595240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:07:32.597947 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.597933 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 05 13:07:32.597998 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.597948 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 05 13:07:32.608115 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.608084 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 05 13:07:32.609777 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.609754 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/840dd6b5-5015-4128-be30-aa29c74fa1e8-original-pull-secret\") pod \"global-pull-secret-syncer-7q58h\" (UID: \"840dd6b5-5015-4128-be30-aa29c74fa1e8\") " pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:07:32.617833 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.617813 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2nsw\" (UniqueName: \"kubernetes.io/projected/8541b5bf-1e1a-4c0a-b395-ef58d894cbe7-kube-api-access-p2nsw\") pod \"network-check-target-2z62p\" (UID: \"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7\") " pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:07:32.689740 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.689718 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q58h" Jun 05 13:07:32.703679 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.703661 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nhxzk\"" Jun 05 13:07:32.712473 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.712457 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:07:32.818264 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.818225 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q58h"] Jun 05 13:07:32.821433 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:07:32.821404 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840dd6b5_5015_4128_be30_aa29c74fa1e8.slice/crio-25bfe474b95dc9deaa0f845dd0f846058a20f2d42bf560de6eba8340d27e0889 WatchSource:0}: Error finding container 25bfe474b95dc9deaa0f845dd0f846058a20f2d42bf560de6eba8340d27e0889: Status 404 returned error can't find the container with id 25bfe474b95dc9deaa0f845dd0f846058a20f2d42bf560de6eba8340d27e0889 Jun 05 13:07:32.835983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:32.835962 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2z62p"] Jun 05 13:07:32.838824 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:07:32.838805 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8541b5bf_1e1a_4c0a_b395_ef58d894cbe7.slice/crio-d80f6461ac30be1f4e4bf1e68de392045c7b2692e357f4c6d054dee861f48e17 WatchSource:0}: Error finding container d80f6461ac30be1f4e4bf1e68de392045c7b2692e357f4c6d054dee861f48e17: Status 404 returned error can't find the container with id d80f6461ac30be1f4e4bf1e68de392045c7b2692e357f4c6d054dee861f48e17 Jun 05 13:07:33.085908 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:33.085831 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2z62p" event={"ID":"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7","Type":"ContainerStarted","Data":"d80f6461ac30be1f4e4bf1e68de392045c7b2692e357f4c6d054dee861f48e17"} Jun 05 13:07:33.086849 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:33.086825 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q58h" event={"ID":"840dd6b5-5015-4128-be30-aa29c74fa1e8","Type":"ContainerStarted","Data":"25bfe474b95dc9deaa0f845dd0f846058a20f2d42bf560de6eba8340d27e0889"} Jun 05 13:07:33.099142 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:33.099125 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:07:33.099201 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:33.099160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:07:33.099304 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:33.099290 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:07:33.099355 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:33.099346 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:08:05.099332351 +0000 UTC m=+97.740656129 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:07:33.099397 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:33.099294 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:07:33.099397 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:33.099390 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:07:33.099457 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:07:33.099446 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:08:05.099432591 +0000 UTC m=+97.740756379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:07:38.100508 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:38.100471 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2z62p" event={"ID":"8541b5bf-1e1a-4c0a-b395-ef58d894cbe7","Type":"ContainerStarted","Data":"8e8baa1205f4cacb5e80937af5631c167d81506427fe52b0a221f25ca6657732"} Jun 05 13:07:38.101018 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:38.100545 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:07:38.101946 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:38.101926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q58h" event={"ID":"840dd6b5-5015-4128-be30-aa29c74fa1e8","Type":"ContainerStarted","Data":"a7db6c62fadc0b5d4679c18ea87c650cf7d5a18efad8cd3d4254caf2a1549a70"} Jun 05 13:07:38.117573 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:38.117530 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2z62p" podStartSLOduration=66.62151535 podStartE2EDuration="1m11.117520135s" totalImagesPullingTime="4.496004785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:07:38.117148058 +0000 UTC m=+70.758471856" watchObservedRunningTime="2026-06-05 13:07:38.117520135 +0000 UTC m=+70.758843928" Jun 05 13:07:38.132202 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:38.132166 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7q58h" podStartSLOduration=65.614613662 podStartE2EDuration="1m10.132153603s" totalImagesPullingTime="4.517539941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:07:38.131504535 +0000 UTC m=+70.772828336" watchObservedRunningTime="2026-06-05 13:07:38.132153603 +0000 UTC m=+70.773477402" Jun 05 13:07:40.868831 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:40.868786 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:40.933396 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:40.933359 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:07:57.302917 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:07:57.302789 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:08:05.108313 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:08:05.108271 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:08:05.108776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:08:05.108322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:08:05.108776 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:05.108396 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:08:05.108776 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:05.108413 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:08:05.108776 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:05.108470 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:09:09.108456401 +0000 UTC m=+161.749780179 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:08:05.108776 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:05.108413 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:08:05.108776 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:05.108576 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:09:09.1085606 +0000 UTC m=+161.749884382 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:08:09.107656 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:08:09.107625 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2z62p" Jun 05 13:08:36.516036 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:08:36.515990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:08:36.516503 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:36.516132 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 05 13:08:36.516503 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:08:36.516189 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs podName:2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2 nodeName:}" failed. No retries permitted until 2026-06-05 13:10:38.516175057 +0000 UTC m=+251.157498835 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs") pod "network-metrics-daemon-8d2pt" (UID: "2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2") : secret "metrics-daemon-secret" not found Jun 05 13:09:04.246264 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:04.246219 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" Jun 05 13:09:04.268362 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:04.268333 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t27mz" podUID="a6d4bb4b-66ac-4457-99e4-1acafbf1d98e" Jun 05 13:09:04.293038 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:04.293013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:09:04.293128 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:04.293013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t27mz" Jun 05 13:09:04.795743 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:04.795689 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8d2pt" podUID="2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2" Jun 05 13:09:09.133439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:09.133408 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:09:09.133831 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:09.133448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:09:09.133831 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:09.133547 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 05 13:09:09.133831 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:09.133551 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 05 13:09:09.133831 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:09.133569 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f876c6468-qmzf4: secret "image-registry-tls" not found Jun 05 13:09:09.133831 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:09.133604 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls podName:a6d4bb4b-66ac-4457-99e4-1acafbf1d98e nodeName:}" failed. No retries permitted until 2026-06-05 13:11:11.133591346 +0000 UTC m=+283.774915124 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls") pod "dns-default-t27mz" (UID: "a6d4bb4b-66ac-4457-99e4-1acafbf1d98e") : secret "dns-default-metrics-tls" not found Jun 05 13:09:09.133831 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:09.133618 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls podName:64c5754e-c750-4fdf-a2e5-19a92fa6671e nodeName:}" failed. No retries permitted until 2026-06-05 13:11:11.133611662 +0000 UTC m=+283.774935440 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls") pod "image-registry-6f876c6468-qmzf4" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e") : secret "image-registry-tls" not found Jun 05 13:09:15.776509 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:15.776468 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:09:47.781884 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.781854 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-tsj2d"] Jun 05 13:09:47.784539 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.784523 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.789336 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.789314 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 05 13:09:47.789457 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.789346 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 05 13:09:47.789654 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.789636 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 05 13:09:47.790038 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.790023 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 05 13:09:47.790086 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.790026 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7qjtz\"" Jun 05 13:09:47.793040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.793021 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr"] Jun 05 13:09:47.795626 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.795606 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" Jun 05 13:09:47.801657 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.801630 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 05 13:09:47.804219 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.804203 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 05 13:09:47.804352 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.804335 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 05 13:09:47.804395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.804371 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ln8d5\"" Jun 05 13:09:47.812439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.812422 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-jjn5c"] Jun 05 13:09:47.815095 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.815080 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.820932 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.820913 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx"] Jun 05 13:09:47.823460 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.823443 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.827582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.827528 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 05 13:09:47.827582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.827528 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 05 13:09:47.827737 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.827677 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 05 13:09:47.831981 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.831962 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-tsj2d"] Jun 05 13:09:47.835754 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.835737 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-s6nm4\"" Jun 05 13:09:47.835860 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.835843 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 05 13:09:47.835964 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.835946 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dm2l8\"" Jun 05 13:09:47.836068 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.835999 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 05 13:09:47.836068 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.836010 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 05 13:09:47.836174 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.836114 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 05 13:09:47.836487 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.836474 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 05 13:09:47.853983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.853956 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 05 13:09:47.854844 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.854795 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr"] Jun 05 13:09:47.855790 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.855769 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-jjn5c"] Jun 05 13:09:47.876576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.876551 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx"] Jun 05 13:09:47.884270 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884233 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpgk\" (UniqueName: \"kubernetes.io/projected/ee227f79-486b-42c0-9500-cbdb8540900c-kube-api-access-jzpgk\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.884385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884286 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.884385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7msp4\" (UniqueName: \"kubernetes.io/projected/295dbce6-3dc9-42bc-b917-8c6f21a339eb-kube-api-access-7msp4\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.884385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884356 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee227f79-486b-42c0-9500-cbdb8540900c-trusted-ca-bundle\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.884385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884374 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75838c9e-e3ab-4c39-848c-528d0dcf1fac-config\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.884513 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884412 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee227f79-486b-42c0-9500-cbdb8540900c-service-ca-bundle\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.884513 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884448 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee227f79-486b-42c0-9500-cbdb8540900c-serving-cert\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.884513 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884465 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f6m5\" (UniqueName: \"kubernetes.io/projected/75838c9e-e3ab-4c39-848c-528d0dcf1fac-kube-api-access-7f6m5\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.884513 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884482 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee227f79-486b-42c0-9500-cbdb8540900c-snapshots\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.884633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75838c9e-e3ab-4c39-848c-528d0dcf1fac-trusted-ca\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.884633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75838c9e-e3ab-4c39-848c-528d0dcf1fac-serving-cert\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.884633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884580 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee227f79-486b-42c0-9500-cbdb8540900c-tmp\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.884633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884596 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv2w4\" (UniqueName: \"kubernetes.io/projected/92e0b03a-57f1-4269-bafb-d857d716d238-kube-api-access-dv2w4\") pod \"volume-data-source-validator-78dd696465-42xdr\" (UID: \"92e0b03a-57f1-4269-bafb-d857d716d238\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" Jun 05 13:09:47.884633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.884617 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/295dbce6-3dc9-42bc-b917-8c6f21a339eb-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.985408 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee227f79-486b-42c0-9500-cbdb8540900c-serving-cert\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.985408 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985408 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f6m5\" (UniqueName: \"kubernetes.io/projected/75838c9e-e3ab-4c39-848c-528d0dcf1fac-kube-api-access-7f6m5\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.985664 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985427 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee227f79-486b-42c0-9500-cbdb8540900c-snapshots\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.985664 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985446 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75838c9e-e3ab-4c39-848c-528d0dcf1fac-trusted-ca\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.985664 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75838c9e-e3ab-4c39-848c-528d0dcf1fac-serving-cert\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.985664 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985496 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee227f79-486b-42c0-9500-cbdb8540900c-tmp\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.985664 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv2w4\" (UniqueName: \"kubernetes.io/projected/92e0b03a-57f1-4269-bafb-d857d716d238-kube-api-access-dv2w4\") pod \"volume-data-source-validator-78dd696465-42xdr\" (UID: \"92e0b03a-57f1-4269-bafb-d857d716d238\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" Jun 05 13:09:47.985907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/295dbce6-3dc9-42bc-b917-8c6f21a339eb-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.985907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpgk\" (UniqueName: \"kubernetes.io/projected/ee227f79-486b-42c0-9500-cbdb8540900c-kube-api-access-jzpgk\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.985907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985735 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.985907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7msp4\" (UniqueName: \"kubernetes.io/projected/295dbce6-3dc9-42bc-b917-8c6f21a339eb-kube-api-access-7msp4\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.986100 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.985982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee227f79-486b-42c0-9500-cbdb8540900c-trusted-ca-bundle\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.986100 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986041 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75838c9e-e3ab-4c39-848c-528d0dcf1fac-config\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.986100 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:47.986074 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:47.986100 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986074 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee227f79-486b-42c0-9500-cbdb8540900c-service-ca-bundle\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.986324 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:47.986137 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls podName:295dbce6-3dc9-42bc-b917-8c6f21a339eb nodeName:}" failed. No retries permitted until 2026-06-05 13:09:48.486122506 +0000 UTC m=+201.127446299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-q2vgx" (UID: "295dbce6-3dc9-42bc-b917-8c6f21a339eb") : secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:47.986324 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986145 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee227f79-486b-42c0-9500-cbdb8540900c-tmp\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.986324 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986272 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee227f79-486b-42c0-9500-cbdb8540900c-snapshots\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.986489 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986396 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/75838c9e-e3ab-4c39-848c-528d0dcf1fac-trusted-ca\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.986776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986747 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee227f79-486b-42c0-9500-cbdb8540900c-service-ca-bundle\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.986888 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986746 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/295dbce6-3dc9-42bc-b917-8c6f21a339eb-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:47.986888 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986758 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75838c9e-e3ab-4c39-848c-528d0dcf1fac-config\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:47.986964 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.986911 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee227f79-486b-42c0-9500-cbdb8540900c-trusted-ca-bundle\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.987868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.987846 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee227f79-486b-42c0-9500-cbdb8540900c-serving-cert\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:47.988180 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:47.988163 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75838c9e-e3ab-4c39-848c-528d0dcf1fac-serving-cert\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:48.007734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.007709 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f6m5\" (UniqueName: \"kubernetes.io/projected/75838c9e-e3ab-4c39-848c-528d0dcf1fac-kube-api-access-7f6m5\") pod \"console-operator-668fdbf85c-jjn5c\" (UID: \"75838c9e-e3ab-4c39-848c-528d0dcf1fac\") " pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:48.010149 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.010122 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7msp4\" (UniqueName: \"kubernetes.io/projected/295dbce6-3dc9-42bc-b917-8c6f21a339eb-kube-api-access-7msp4\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:48.011749 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.011729 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv2w4\" (UniqueName: \"kubernetes.io/projected/92e0b03a-57f1-4269-bafb-d857d716d238-kube-api-access-dv2w4\") pod \"volume-data-source-validator-78dd696465-42xdr\" (UID: \"92e0b03a-57f1-4269-bafb-d857d716d238\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" Jun 05 13:09:48.014175 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.014157 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpgk\" (UniqueName: \"kubernetes.io/projected/ee227f79-486b-42c0-9500-cbdb8540900c-kube-api-access-jzpgk\") pod \"insights-operator-896549976-tsj2d\" (UID: \"ee227f79-486b-42c0-9500-cbdb8540900c\") " pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:48.027966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.027947 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw"] Jun 05 13:09:48.032077 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.032042 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" Jun 05 13:09:48.034185 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.034161 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"network-diagnostics-dockercfg-9xxg6\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-diagnostics\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9xxg6\"" type="*v1.Secret" Jun 05 13:09:48.075538 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.075518 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw"] Jun 05 13:09:48.086574 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.086555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fhl8\" (UniqueName: \"kubernetes.io/projected/8a30b113-fa89-4e34-8628-fe6a1530aa06-kube-api-access-5fhl8\") pod \"network-check-source-dbd9cf844-fswqw\" (UID: \"8a30b113-fa89-4e34-8628-fe6a1530aa06\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" Jun 05 13:09:48.093590 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.093574 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-tsj2d" Jun 05 13:09:48.098199 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.098181 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6"] Jun 05 13:09:48.101103 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.101088 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.104287 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.104265 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" Jun 05 13:09:48.107692 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.107673 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 05 13:09:48.107788 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.107675 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 05 13:09:48.107788 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.107721 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 05 13:09:48.107788 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.107733 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-f799b\"" Jun 05 13:09:48.107946 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.107677 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 05 13:09:48.122649 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.122627 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:48.134538 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.134515 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6"] Jun 05 13:09:48.152140 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.152019 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g"] Jun 05 13:09:48.155770 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.155622 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:48.170457 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.170325 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-s78zq\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s78zq\"" type="*v1.Secret" Jun 05 13:09:48.170457 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.170383 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" Jun 05 13:09:48.170457 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.170420 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" Jun 05 13:09:48.176022 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.175961 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g"] Jun 05 13:09:48.178500 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.178467 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"networking-console-plugin-d5d5bb78b-lpd2g\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" podUID="611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:48.187915 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.187890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fhl8\" (UniqueName: \"kubernetes.io/projected/8a30b113-fa89-4e34-8628-fe6a1530aa06-kube-api-access-5fhl8\") pod \"network-check-source-dbd9cf844-fswqw\" (UID: \"8a30b113-fa89-4e34-8628-fe6a1530aa06\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" Jun 05 13:09:48.188022 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.187935 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1a06d16-edfd-48d5-9168-0cd21d991511-config\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.188080 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.188031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:48.188080 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.188059 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:48.188181 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.188117 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a06d16-edfd-48d5-9168-0cd21d991511-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.188181 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.188151 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgwmd\" (UniqueName: \"kubernetes.io/projected/e1a06d16-edfd-48d5-9168-0cd21d991511-kube-api-access-mgwmd\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.247265 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.247223 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fhl8\" (UniqueName: \"kubernetes.io/projected/8a30b113-fa89-4e34-8628-fe6a1530aa06-kube-api-access-5fhl8\") pod \"network-check-source-dbd9cf844-fswqw\" (UID: \"8a30b113-fa89-4e34-8628-fe6a1530aa06\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" Jun 05 13:09:48.278417 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.278392 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr"] Jun 05 13:09:48.285143 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.285087 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-tsj2d"] Jun 05 13:09:48.287453 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.287434 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-jjn5c"] Jun 05 13:09:48.288479 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:09:48.288453 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee227f79_486b_42c0_9500_cbdb8540900c.slice/crio-41098247ff5fd41a5e211319fc044dd5510f3080f8bd81259e4d3f82cc95d01e WatchSource:0}: Error finding container 41098247ff5fd41a5e211319fc044dd5510f3080f8bd81259e4d3f82cc95d01e: Status 404 returned error can't find the container with id 41098247ff5fd41a5e211319fc044dd5510f3080f8bd81259e4d3f82cc95d01e Jun 05 13:09:48.288623 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.288489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:48.288623 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.288604 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:48.288818 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.288640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a06d16-edfd-48d5-9168-0cd21d991511-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.288818 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.288663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgwmd\" (UniqueName: \"kubernetes.io/projected/e1a06d16-edfd-48d5-9168-0cd21d991511-kube-api-access-mgwmd\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.288818 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.288704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1a06d16-edfd-48d5-9168-0cd21d991511-config\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.289262 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:09:48.289226 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75838c9e_e3ab_4c39_848c_528d0dcf1fac.slice/crio-95d8f2a6de656d7cbfbc4f38f3ef08bbbaca20216df6ba75a4b299af37257278 WatchSource:0}: Error finding container 95d8f2a6de656d7cbfbc4f38f3ef08bbbaca20216df6ba75a4b299af37257278: Status 404 returned error can't find the container with id 95d8f2a6de656d7cbfbc4f38f3ef08bbbaca20216df6ba75a4b299af37257278 Jun 05 13:09:48.289378 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.289327 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1a06d16-edfd-48d5-9168-0cd21d991511-config\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.290650 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:09:48.290621 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e0b03a_57f1_4269_bafb_d857d716d238.slice/crio-31c1b083d00aecb896544fb3a26250af7250dacdca33439aba9c854068ba62ea WatchSource:0}: Error finding container 31c1b083d00aecb896544fb3a26250af7250dacdca33439aba9c854068ba62ea: Status 404 returned error can't find the container with id 31c1b083d00aecb896544fb3a26250af7250dacdca33439aba9c854068ba62ea Jun 05 13:09:48.291078 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.290996 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1a06d16-edfd-48d5-9168-0cd21d991511-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.308536 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.308518 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgwmd\" (UniqueName: \"kubernetes.io/projected/e1a06d16-edfd-48d5-9168-0cd21d991511-kube-api-access-mgwmd\") pod \"service-ca-operator-6bc5ff68f5-mkvx6\" (UID: \"e1a06d16-edfd-48d5-9168-0cd21d991511\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.391334 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.391305 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" event={"ID":"75838c9e-e3ab-4c39-848c-528d0dcf1fac","Type":"ContainerStarted","Data":"95d8f2a6de656d7cbfbc4f38f3ef08bbbaca20216df6ba75a4b299af37257278"} Jun 05 13:09:48.392198 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.392174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-tsj2d" event={"ID":"ee227f79-486b-42c0-9500-cbdb8540900c","Type":"ContainerStarted","Data":"41098247ff5fd41a5e211319fc044dd5510f3080f8bd81259e4d3f82cc95d01e"} Jun 05 13:09:48.393042 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.393025 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" event={"ID":"92e0b03a-57f1-4269-bafb-d857d716d238","Type":"ContainerStarted","Data":"31c1b083d00aecb896544fb3a26250af7250dacdca33439aba9c854068ba62ea"} Jun 05 13:09:48.409562 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.409544 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" Jun 05 13:09:48.489707 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.489678 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:48.489830 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.489814 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:48.489888 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:48.489878 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls podName:295dbce6-3dc9-42bc-b917-8c6f21a339eb nodeName:}" failed. No retries permitted until 2026-06-05 13:09:49.489862044 +0000 UTC m=+202.131185833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-q2vgx" (UID: "295dbce6-3dc9-42bc-b917-8c6f21a339eb") : secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:48.560574 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.560496 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6"] Jun 05 13:09:48.562945 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:09:48.562923 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a06d16_edfd_48d5_9168_0cd21d991511.slice/crio-4104ce849d953f6342cd4b91547127d90675a35db4c1aec0f39c16564fd2ba98 WatchSource:0}: Error finding container 4104ce849d953f6342cd4b91547127d90675a35db4c1aec0f39c16564fd2ba98: Status 404 returned error can't find the container with id 4104ce849d953f6342cd4b91547127d90675a35db4c1aec0f39c16564fd2ba98 Jun 05 13:09:48.675020 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.674991 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:48.831920 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:48.831840 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:49.073600 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.073515 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 05 13:09:49.079908 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.079854 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:49.144681 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.144592 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s78zq\"" Jun 05 13:09:49.289065 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:49.289038 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: failed to sync secret cache: timed out waiting for the condition Jun 05 13:09:49.289221 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:49.289130 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert podName:611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac nodeName:}" failed. No retries permitted until 2026-06-05 13:09:49.789107725 +0000 UTC m=+202.430431508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-lpd2g" (UID: "611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac") : failed to sync secret cache: timed out waiting for the condition Jun 05 13:09:49.340808 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.340592 2570 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jun 05 13:09:49.340808 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.340649 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" Jun 05 13:09:49.366700 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.366672 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9xxg6\"" Jun 05 13:09:49.400020 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.399930 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" event={"ID":"e1a06d16-edfd-48d5-9168-0cd21d991511","Type":"ContainerStarted","Data":"4104ce849d953f6342cd4b91547127d90675a35db4c1aec0f39c16564fd2ba98"} Jun 05 13:09:49.431115 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.431087 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 05 13:09:49.497758 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.497719 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:49.497917 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:49.497903 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:49.497994 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:49.497983 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls podName:295dbce6-3dc9-42bc-b917-8c6f21a339eb nodeName:}" failed. No retries permitted until 2026-06-05 13:09:51.497964909 +0000 UTC m=+204.139288693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-q2vgx" (UID: "295dbce6-3dc9-42bc-b917-8c6f21a339eb") : secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:49.521019 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.519693 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw"] Jun 05 13:09:49.674570 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:09:49.674489 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a30b113_fa89_4e34_8628_fe6a1530aa06.slice/crio-1b9a8ab5d903a84cb9f0023209652a5f4d62cb9eefc009fc4bef66ad6a3295dc WatchSource:0}: Error finding container 1b9a8ab5d903a84cb9f0023209652a5f4d62cb9eefc009fc4bef66ad6a3295dc: Status 404 returned error can't find the container with id 1b9a8ab5d903a84cb9f0023209652a5f4d62cb9eefc009fc4bef66ad6a3295dc Jun 05 13:09:49.800113 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:49.800012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:49.800219 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:49.800182 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 05 13:09:49.800358 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:49.800263 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert podName:611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac nodeName:}" failed. No retries permitted until 2026-06-05 13:09:50.800228693 +0000 UTC m=+203.441552495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-lpd2g" (UID: "611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac") : secret "networking-console-plugin-cert" not found Jun 05 13:09:50.405624 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.405378 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" event={"ID":"92e0b03a-57f1-4269-bafb-d857d716d238","Type":"ContainerStarted","Data":"ef2546d77b4710ecf64773ed0de08893f3d10234a888cbb0028c01866bd2b16f"} Jun 05 13:09:50.407364 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.407292 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" event={"ID":"8a30b113-fa89-4e34-8628-fe6a1530aa06","Type":"ContainerStarted","Data":"a4db86911d5cf05b23c9be3193ba057609e565565f343f2e08b5b3ad75c3a468"} Jun 05 13:09:50.407364 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.407337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" event={"ID":"8a30b113-fa89-4e34-8628-fe6a1530aa06","Type":"ContainerStarted","Data":"1b9a8ab5d903a84cb9f0023209652a5f4d62cb9eefc009fc4bef66ad6a3295dc"} Jun 05 13:09:50.437201 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.437039 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-42xdr" podStartSLOduration=2.001589109 podStartE2EDuration="3.437020514s" totalImagesPullingTime="1.435431405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:09:50.436073195 +0000 UTC m=+203.077396996" watchObservedRunningTime="2026-06-05 13:09:50.437020514 +0000 UTC m=+203.078344313" Jun 05 13:09:50.502639 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.502582 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-fswqw" podStartSLOduration=3.5025630420000002 podStartE2EDuration="3.502563042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:09:50.502081992 +0000 UTC m=+203.143405793" watchObservedRunningTime="2026-06-05 13:09:50.502563042 +0000 UTC m=+203.143886843" Jun 05 13:09:50.577176 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.577142 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:50.689963 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.689892 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:50.807554 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:50.807508 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:50.807716 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:50.807668 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 05 13:09:50.807764 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:50.807735 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert podName:611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac nodeName:}" failed. No retries permitted until 2026-06-05 13:09:52.807717337 +0000 UTC m=+205.449041116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-lpd2g" (UID: "611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac") : secret "networking-console-plugin-cert" not found Jun 05 13:09:51.514084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:51.514044 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:51.514524 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:51.514218 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:51.514524 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:51.514316 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls podName:295dbce6-3dc9-42bc-b917-8c6f21a339eb nodeName:}" failed. No retries permitted until 2026-06-05 13:09:55.514294475 +0000 UTC m=+208.155618270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-q2vgx" (UID: "295dbce6-3dc9-42bc-b917-8c6f21a339eb") : secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:52.413849 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.413824 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/0.log" Jun 05 13:09:52.414022 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.413860 2570 generic.go:356] "Generic (PLEG): container finished" podID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" containerID="3e77a94a4ee87df7f6721120fc3ccb4de352d69ced8df022ae58802364f98b18" exitCode=255 Jun 05 13:09:52.414022 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.413924 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" event={"ID":"75838c9e-e3ab-4c39-848c-528d0dcf1fac","Type":"ContainerDied","Data":"3e77a94a4ee87df7f6721120fc3ccb4de352d69ced8df022ae58802364f98b18"} Jun 05 13:09:52.414155 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.414139 2570 scope.go:117] "RemoveContainer" containerID="3e77a94a4ee87df7f6721120fc3ccb4de352d69ced8df022ae58802364f98b18" Jun 05 13:09:52.415490 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.415464 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-tsj2d" event={"ID":"ee227f79-486b-42c0-9500-cbdb8540900c","Type":"ContainerStarted","Data":"314a144230fac56370430773231e7c6c86c0944a47385f34797479179088c3a3"} Jun 05 13:09:52.416902 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.416878 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" event={"ID":"e1a06d16-edfd-48d5-9168-0cd21d991511","Type":"ContainerStarted","Data":"736b719f97e1b822efc38418ccff47cfcd0afba4d9c271a04b0734a0e2613696"} Jun 05 13:09:52.486627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.486576 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-tsj2d" podStartSLOduration=2.059457439 podStartE2EDuration="5.486557759s" totalImagesPullingTime="3.42710032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:09:52.480831987 +0000 UTC m=+205.122155789" watchObservedRunningTime="2026-06-05 13:09:52.486557759 +0000 UTC m=+205.127881559" Jun 05 13:09:52.539062 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.539013 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" podStartSLOduration=2.380455249 podStartE2EDuration="5.538993501s" totalImagesPullingTime="3.158538252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:09:52.537860253 +0000 UTC m=+205.179184055" watchObservedRunningTime="2026-06-05 13:09:52.538993501 +0000 UTC m=+205.180317303" Jun 05 13:09:52.824989 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:52.824959 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:52.825101 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:52.825065 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 05 13:09:52.825139 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:52.825113 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert podName:611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac nodeName:}" failed. No retries permitted until 2026-06-05 13:09:56.825100202 +0000 UTC m=+209.466423981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-lpd2g" (UID: "611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac") : secret "networking-console-plugin-cert" not found Jun 05 13:09:53.332523 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.332479 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:53.421569 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.421541 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/1.log" Jun 05 13:09:53.421924 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.421909 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/0.log" Jun 05 13:09:53.421971 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.421946 2570 generic.go:356] "Generic (PLEG): container finished" podID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" exitCode=255 Jun 05 13:09:53.422058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.422039 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" event={"ID":"75838c9e-e3ab-4c39-848c-528d0dcf1fac","Type":"ContainerDied","Data":"6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b"} Jun 05 13:09:53.422130 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.422119 2570 scope.go:117] "RemoveContainer" containerID="3e77a94a4ee87df7f6721120fc3ccb4de352d69ced8df022ae58802364f98b18" Jun 05 13:09:53.422367 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:53.422347 2570 scope.go:117] "RemoveContainer" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" Jun 05 13:09:53.422540 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:53.422521 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:09:54.426720 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:54.426694 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/1.log" Jun 05 13:09:54.427179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:54.427067 2570 scope.go:117] "RemoveContainer" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" Jun 05 13:09:54.427301 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:54.427279 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:09:54.913139 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:54.913106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:55.544287 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.544225 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:09:55.544718 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:55.544402 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:55.544718 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:55.544487 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls podName:295dbce6-3dc9-42bc-b917-8c6f21a339eb nodeName:}" failed. No retries permitted until 2026-06-05 13:10:03.54446619 +0000 UTC m=+216.185789989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-q2vgx" (UID: "295dbce6-3dc9-42bc-b917-8c6f21a339eb") : secret "cluster-monitoring-operator-tls" not found Jun 05 13:09:55.580529 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.580504 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-xm6k9"] Jun 05 13:09:55.583405 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.583389 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.586064 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.586042 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 05 13:09:55.586960 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.586942 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 05 13:09:55.586960 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.586952 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-26n8z\"" Jun 05 13:09:55.587110 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.586971 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 05 13:09:55.587110 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.587006 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 05 13:09:55.610350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.610329 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-xm6k9"] Jun 05 13:09:55.644886 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.644868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-signing-cabundle\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.644977 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.644898 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-signing-key\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.644977 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.644923 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5955p\" (UniqueName: \"kubernetes.io/projected/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-kube-api-access-5955p\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.745768 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.745741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-signing-cabundle\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.745880 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.745776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-signing-key\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.745880 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.745805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5955p\" (UniqueName: \"kubernetes.io/projected/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-kube-api-access-5955p\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.746503 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.746486 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-signing-cabundle\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.748024 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.748004 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-signing-key\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.765258 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.765225 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5955p\" (UniqueName: \"kubernetes.io/projected/bdf071ee-f9b1-44a0-9ec4-15869ac934c8-kube-api-access-5955p\") pod \"service-ca-7d74d578b8-xm6k9\" (UID: \"bdf071ee-f9b1-44a0-9ec4-15869ac934c8\") " pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.779737 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.779718 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:55.891801 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.891740 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" Jun 05 13:09:55.915610 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:55.915580 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:56.038655 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.038622 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-xm6k9"] Jun 05 13:09:56.040994 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:09:56.040968 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf071ee_f9b1_44a0_9ec4_15869ac934c8.slice/crio-de6b1f51de64ad9ca44490664e365240ea6fb11516aea57f8dceeccdf15f382c WatchSource:0}: Error finding container de6b1f51de64ad9ca44490664e365240ea6fb11516aea57f8dceeccdf15f382c: Status 404 returned error can't find the container with id de6b1f51de64ad9ca44490664e365240ea6fb11516aea57f8dceeccdf15f382c Jun 05 13:09:56.363161 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.363137 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2d6ll_e3e1da7e-c5f3-4a04-95c8-230051ddcafe/dns-node-resolver/0.log" Jun 05 13:09:56.434960 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.434925 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" event={"ID":"bdf071ee-f9b1-44a0-9ec4-15869ac934c8","Type":"ContainerStarted","Data":"05346503743e4ff2dfa7709418a5389bd921c51e88b6bd550cd8d9c091a3ed94"} Jun 05 13:09:56.435081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.434980 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" event={"ID":"bdf071ee-f9b1-44a0-9ec4-15869ac934c8","Type":"ContainerStarted","Data":"de6b1f51de64ad9ca44490664e365240ea6fb11516aea57f8dceeccdf15f382c"} Jun 05 13:09:56.469151 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.469125 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:56.486556 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.486514 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-xm6k9" podStartSLOduration=1.486501495 podStartE2EDuration="1.486501495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:09:56.485921852 +0000 UTC m=+209.127245651" watchObservedRunningTime="2026-06-05 13:09:56.486501495 +0000 UTC m=+209.127825294" Jun 05 13:09:56.830756 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.830720 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:56.854971 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:56.854946 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:09:56.855104 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:56.855083 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 05 13:09:56.855149 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:56.855139 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert podName:611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac nodeName:}" failed. No retries permitted until 2026-06-05 13:10:04.855124471 +0000 UTC m=+217.496448249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-lpd2g" (UID: "611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac") : secret "networking-console-plugin-cert" not found Jun 05 13:09:57.126231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:57.126145 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:57.395466 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:57.395366 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:57.535215 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:57.535180 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:57.566061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:57.566036 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zg2xk_3a2c2470-f752-457d-b759-fad45632ee5f/node-ca/0.log" Jun 05 13:09:57.772927 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:57.772887 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:58.093575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.093502 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:58.123434 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.123411 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:58.123493 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.123482 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:09:58.123762 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.123749 2570 scope.go:117] "RemoveContainer" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" Jun 05 13:09:58.123903 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:58.123886 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:09:58.145937 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.145910 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d2nvt_f5dcc392-3021-4ab6-8b13-a85ec0cfedf4/serve-healthcheck-canary/0.log" Jun 05 13:09:58.441995 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.441927 2570 scope.go:117] "RemoveContainer" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" Jun 05 13:09:58.442136 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:09:58.442065 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:09:58.829066 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:58.829031 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:59.109121 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:59.109038 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:59.309084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:59.309046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:59.432153 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:59.432077 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:59.578581 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:59.578546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:09:59.811973 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:09:59.811937 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:00.961208 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:00.961178 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:01.166877 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:01.166842 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:01.402431 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:01.402400 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:01.600294 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:01.600242 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:01.784433 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:01.784404 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:02.986741 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:02.986707 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:03.238163 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:03.238069 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:03.361518 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:03.361480 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:03.545776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:03.545695 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:03.616001 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:03.615968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:10:03.616178 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:03.616116 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 05 13:10:03.616219 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:03.616179 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls podName:295dbce6-3dc9-42bc-b917-8c6f21a339eb nodeName:}" failed. No retries permitted until 2026-06-05 13:10:19.616163838 +0000 UTC m=+232.257487616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-q2vgx" (UID: "295dbce6-3dc9-42bc-b917-8c6f21a339eb") : secret "cluster-monitoring-operator-tls" not found Jun 05 13:10:03.744603 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:03.744573 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:03.911536 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:03.911446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:04.576867 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:04.576837 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:04.928531 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:04.928454 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:10:04.930671 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:04.930650 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-lpd2g\" (UID: \"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:10:04.970668 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:04.970642 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" Jun 05 13:10:05.115731 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:05.115707 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g"] Jun 05 13:10:05.118169 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:05.118143 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611f6ea8_f9d0_47dc_8c94_6f1d1653e6ac.slice/crio-421c0ffbe8edf94a803aa45e8492409740d6c83aa1caa8754ce3dc997dd38f44 WatchSource:0}: Error finding container 421c0ffbe8edf94a803aa45e8492409740d6c83aa1caa8754ce3dc997dd38f44: Status 404 returned error can't find the container with id 421c0ffbe8edf94a803aa45e8492409740d6c83aa1caa8754ce3dc997dd38f44 Jun 05 13:10:05.209031 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:05.208965 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:05.462736 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:05.462662 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" event={"ID":"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac","Type":"ContainerStarted","Data":"421c0ffbe8edf94a803aa45e8492409740d6c83aa1caa8754ce3dc997dd38f44"} Jun 05 13:10:05.501098 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:05.501065 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:05.744005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:05.743962 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:06.099734 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:06.099652 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:06.565020 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:06.564997 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:06.892127 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:06.892053 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:07.185141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:07.185066 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:07.474105 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:07.474016 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" event={"ID":"611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac","Type":"ContainerStarted","Data":"8c42dd55197742cabf6f7b85d29e7269ee112d5bcd940739c3a50b0cec9def10"} Jun 05 13:10:07.532546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:07.532520 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:07.683052 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:07.683018 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:07.812024 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:07.811996 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:08.786664 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:08.786632 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:09.118416 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:09.118336 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:10.099576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:10.099545 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:10.777037 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:10.777008 2570 scope.go:117] "RemoveContainer" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" Jun 05 13:10:11.486369 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.486339 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:10:11.486737 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.486704 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/1.log" Jun 05 13:10:11.486782 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.486732 2570 generic.go:356] "Generic (PLEG): container finished" podID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" containerID="602788f06ea4cad55d2484709f3c92c7f2826f220cefe08dcbe01ebe066ff39c" exitCode=255 Jun 05 13:10:11.486815 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.486781 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" event={"ID":"75838c9e-e3ab-4c39-848c-528d0dcf1fac","Type":"ContainerDied","Data":"602788f06ea4cad55d2484709f3c92c7f2826f220cefe08dcbe01ebe066ff39c"} Jun 05 13:10:11.486849 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.486824 2570 scope.go:117] "RemoveContainer" containerID="6b638a9613a4ba0af85e94e66675451f224f6a1ff9c2c203f15b20334030776b" Jun 05 13:10:11.487137 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.487120 2570 scope.go:117] "RemoveContainer" containerID="602788f06ea4cad55d2484709f3c92c7f2826f220cefe08dcbe01ebe066ff39c" Jun 05 13:10:11.487300 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:11.487282 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:10:11.556349 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:11.556298 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-lpd2g" podStartSLOduration=22.143759869 podStartE2EDuration="23.556283915s" totalImagesPullingTime="1.412524046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:07.524619141 +0000 UTC m=+220.165942940" watchObservedRunningTime="2026-06-05 13:10:11.556283915 +0000 UTC m=+224.197607723" Jun 05 13:10:12.491222 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:12.491195 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:10:16.293751 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:16.293718 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:18.123707 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:18.123662 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:10:18.124133 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:18.123732 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:10:18.124133 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:18.124037 2570 scope.go:117] "RemoveContainer" containerID="602788f06ea4cad55d2484709f3c92c7f2826f220cefe08dcbe01ebe066ff39c" Jun 05 13:10:18.124229 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:18.124209 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:10:19.643488 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:19.643448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:10:19.645825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:19.645805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/295dbce6-3dc9-42bc-b917-8c6f21a339eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-q2vgx\" (UID: \"295dbce6-3dc9-42bc-b917-8c6f21a339eb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:10:19.934838 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:19.934757 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" Jun 05 13:10:20.090913 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:20.090874 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295dbce6_3dc9_42bc_b917_8c6f21a339eb.slice/crio-fc86169ef0e4ccf61e490b7cb60c085cf95c31a2a03c3794238090b38512b9df WatchSource:0}: Error finding container fc86169ef0e4ccf61e490b7cb60c085cf95c31a2a03c3794238090b38512b9df: Status 404 returned error can't find the container with id fc86169ef0e4ccf61e490b7cb60c085cf95c31a2a03c3794238090b38512b9df Jun 05 13:10:20.092680 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:20.092657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx"] Jun 05 13:10:20.516122 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:20.516075 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" event={"ID":"295dbce6-3dc9-42bc-b917-8c6f21a339eb","Type":"ContainerStarted","Data":"fc86169ef0e4ccf61e490b7cb60c085cf95c31a2a03c3794238090b38512b9df"} Jun 05 13:10:22.428193 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:22.428146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:22.523414 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:22.523382 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" event={"ID":"295dbce6-3dc9-42bc-b917-8c6f21a339eb","Type":"ContainerStarted","Data":"d989952201353ae5c6e89a4e3345871882c805d02faa1647f36160ea9fc0b2b0"} Jun 05 13:10:22.585076 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:22.585033 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-q2vgx" podStartSLOduration=33.878918307 podStartE2EDuration="35.585019179s" totalImagesPullingTime="1.706100872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:22.584690265 +0000 UTC m=+235.226014065" watchObservedRunningTime="2026-06-05 13:10:22.585019179 +0000 UTC m=+235.226343020" Jun 05 13:10:22.779026 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:22.778989 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:22.968068 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:22.968041 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:26.629489 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.629453 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nl9z6"] Jun 05 13:10:26.634694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.634675 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.653212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.653192 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f2d46\"" Jun 05 13:10:26.654488 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.654473 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 05 13:10:26.658558 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.658541 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 05 13:10:26.682353 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.682332 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9"] Jun 05 13:10:26.684793 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.684779 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:26.694592 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.694575 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vln5b\"" Jun 05 13:10:26.694743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.694724 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 05 13:10:26.800856 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.800831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32d63087-1325-4157-82dd-8e570421d61e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.800967 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.800867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldqzd\" (UniqueName: \"kubernetes.io/projected/32d63087-1325-4157-82dd-8e570421d61e-kube-api-access-ldqzd\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.800967 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.800892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4239b006-96bc-4178-a4f6-a9548fd18b91-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-rcgd9\" (UID: \"4239b006-96bc-4178-a4f6-a9548fd18b91\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:26.800967 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.800912 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32d63087-1325-4157-82dd-8e570421d61e-crio-socket\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.801085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.800967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32d63087-1325-4157-82dd-8e570421d61e-data-volume\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.801085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.800991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32d63087-1325-4157-82dd-8e570421d61e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.807984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.807964 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9"] Jun 05 13:10:26.886011 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.885946 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nl9z6"] Jun 05 13:10:26.901723 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.901704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32d63087-1325-4157-82dd-8e570421d61e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.901817 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.901736 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldqzd\" (UniqueName: \"kubernetes.io/projected/32d63087-1325-4157-82dd-8e570421d61e-kube-api-access-ldqzd\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.901817 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.901763 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4239b006-96bc-4178-a4f6-a9548fd18b91-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-rcgd9\" (UID: \"4239b006-96bc-4178-a4f6-a9548fd18b91\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:26.901817 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.901810 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32d63087-1325-4157-82dd-8e570421d61e-crio-socket\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.901968 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.901833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32d63087-1325-4157-82dd-8e570421d61e-data-volume\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.901968 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.901917 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32d63087-1325-4157-82dd-8e570421d61e-crio-socket\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.902048 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.902033 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32d63087-1325-4157-82dd-8e570421d61e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.902106 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.902092 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32d63087-1325-4157-82dd-8e570421d61e-data-volume\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.902569 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.902545 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32d63087-1325-4157-82dd-8e570421d61e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.903996 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.903975 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32d63087-1325-4157-82dd-8e570421d61e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.904176 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.904160 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4239b006-96bc-4178-a4f6-a9548fd18b91-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-rcgd9\" (UID: \"4239b006-96bc-4178-a4f6-a9548fd18b91\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:26.924688 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.924666 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldqzd\" (UniqueName: \"kubernetes.io/projected/32d63087-1325-4157-82dd-8e570421d61e-kube-api-access-ldqzd\") pod \"insights-runtime-extractor-nl9z6\" (UID: \"32d63087-1325-4157-82dd-8e570421d61e\") " pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.942630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.942608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nl9z6" Jun 05 13:10:26.992204 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:26.992111 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:27.330150 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.330116 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nl9z6"] Jun 05 13:10:27.333918 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:27.333890 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d63087_1325_4157_82dd_8e570421d61e.slice/crio-8f45b343b19a0db45199e8d92a7ccc96a5992264fffcb6df45117fff2bd7609e WatchSource:0}: Error finding container 8f45b343b19a0db45199e8d92a7ccc96a5992264fffcb6df45117fff2bd7609e: Status 404 returned error can't find the container with id 8f45b343b19a0db45199e8d92a7ccc96a5992264fffcb6df45117fff2bd7609e Jun 05 13:10:27.399608 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:27.399581 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4239b006_96bc_4178_a4f6_a9548fd18b91.slice/crio-c8e60197309adc554481b5b20a99016183bc1f84092af67f69875dd7d5100423 WatchSource:0}: Error finding container c8e60197309adc554481b5b20a99016183bc1f84092af67f69875dd7d5100423: Status 404 returned error can't find the container with id c8e60197309adc554481b5b20a99016183bc1f84092af67f69875dd7d5100423 Jun 05 13:10:27.401087 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.401065 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9"] Jun 05 13:10:27.409177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.409157 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:27.539857 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.539826 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nl9z6" event={"ID":"32d63087-1325-4157-82dd-8e570421d61e","Type":"ContainerStarted","Data":"b7e5058b758a959e680e5fa9dc020f46d3744839626b601b9a23b08e08282d68"} Jun 05 13:10:27.540019 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.539880 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nl9z6" event={"ID":"32d63087-1325-4157-82dd-8e570421d61e","Type":"ContainerStarted","Data":"8f45b343b19a0db45199e8d92a7ccc96a5992264fffcb6df45117fff2bd7609e"} Jun 05 13:10:27.540940 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.540920 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" event={"ID":"4239b006-96bc-4178-a4f6-a9548fd18b91","Type":"ContainerStarted","Data":"c8e60197309adc554481b5b20a99016183bc1f84092af67f69875dd7d5100423"} Jun 05 13:10:27.893627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:27.893590 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:28.546131 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:28.546098 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nl9z6" event={"ID":"32d63087-1325-4157-82dd-8e570421d61e","Type":"ContainerStarted","Data":"4858736b2a782848b7917187415edcc8d6da5c1adb85644f18bb3253e0e46c92"} Jun 05 13:10:29.142494 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:29.142461 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:29.551101 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:29.551066 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" event={"ID":"4239b006-96bc-4178-a4f6-a9548fd18b91","Type":"ContainerStarted","Data":"9fdce89f4f71084667c62f45ade2e962b98b87a6814a383404929981cb458d3e"} Jun 05 13:10:29.551317 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:29.551292 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:29.555586 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:29.555562 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" Jun 05 13:10:29.588224 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:29.588181 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rcgd9" podStartSLOduration=2.391411516 podStartE2EDuration="3.588169613s" totalImagesPullingTime="1.196758097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:29.587089646 +0000 UTC m=+242.228413445" watchObservedRunningTime="2026-06-05 13:10:29.588169613 +0000 UTC m=+242.229493391" Jun 05 13:10:29.965304 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:29.965208 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:30.162060 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.162026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:30.189577 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.189550 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7"] Jun 05 13:10:30.192767 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.192753 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.205049 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.205028 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 05 13:10:30.205141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.205029 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 05 13:10:30.205141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.205081 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wqgcz\"" Jun 05 13:10:30.205233 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.205148 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 05 13:10:30.237363 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.237341 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7"] Jun 05 13:10:30.337435 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.337407 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d2361f9-9279-41de-b218-d4d64d23699c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.337575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.337446 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d2361f9-9279-41de-b218-d4d64d23699c-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.337575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.337504 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vzzt\" (UniqueName: \"kubernetes.io/projected/0d2361f9-9279-41de-b218-d4d64d23699c-kube-api-access-5vzzt\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.337575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.337540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2361f9-9279-41de-b218-d4d64d23699c-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.438442 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.438405 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d2361f9-9279-41de-b218-d4d64d23699c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.438608 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.438451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d2361f9-9279-41de-b218-d4d64d23699c-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.438608 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.438488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzzt\" (UniqueName: \"kubernetes.io/projected/0d2361f9-9279-41de-b218-d4d64d23699c-kube-api-access-5vzzt\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.438608 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.438515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2361f9-9279-41de-b218-d4d64d23699c-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.439156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.439132 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d2361f9-9279-41de-b218-d4d64d23699c-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.440774 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.440752 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2361f9-9279-41de-b218-d4d64d23699c-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.440881 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.440761 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d2361f9-9279-41de-b218-d4d64d23699c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.455969 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.455950 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vzzt\" (UniqueName: \"kubernetes.io/projected/0d2361f9-9279-41de-b218-d4d64d23699c-kube-api-access-5vzzt\") pod \"prometheus-operator-7f7d445d84-ggbb7\" (UID: \"0d2361f9-9279-41de-b218-d4d64d23699c\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.464108 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.464088 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:30.501032 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.500983 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" Jun 05 13:10:30.562428 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.562359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nl9z6" event={"ID":"32d63087-1325-4157-82dd-8e570421d61e","Type":"ContainerStarted","Data":"ce8d119d32a43631d347916ccaacc8509e461c53a39f8af83c9d936885bc6e2b"} Jun 05 13:10:30.645619 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.645568 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nl9z6" podStartSLOduration=2.490127515 podStartE2EDuration="4.645550977s" totalImagesPullingTime="2.155423462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:10:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:30.613108863 +0000 UTC m=+243.254432686" watchObservedRunningTime="2026-06-05 13:10:30.645550977 +0000 UTC m=+243.286874776" Jun 05 13:10:30.646379 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.646168 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7"] Jun 05 13:10:30.648612 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:30.648586 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2361f9_9279_41de_b218_d4d64d23699c.slice/crio-b56a3a1162282169ae1a5abbafb49e26b34d3b56fd0eabee78ea8c0d3798b50d WatchSource:0}: Error finding container b56a3a1162282169ae1a5abbafb49e26b34d3b56fd0eabee78ea8c0d3798b50d: Status 404 returned error can't find the container with id b56a3a1162282169ae1a5abbafb49e26b34d3b56fd0eabee78ea8c0d3798b50d Jun 05 13:10:30.777074 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:30.777011 2570 scope.go:117] "RemoveContainer" containerID="602788f06ea4cad55d2484709f3c92c7f2826f220cefe08dcbe01ebe066ff39c" Jun 05 13:10:30.777191 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:30.777145 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-jjn5c_openshift-console-operator(75838c9e-e3ab-4c39-848c-528d0dcf1fac)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podUID="75838c9e-e3ab-4c39-848c-528d0dcf1fac" Jun 05 13:10:31.566859 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:31.566812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" event={"ID":"0d2361f9-9279-41de-b218-d4d64d23699c","Type":"ContainerStarted","Data":"b56a3a1162282169ae1a5abbafb49e26b34d3b56fd0eabee78ea8c0d3798b50d"} Jun 05 13:10:31.602646 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:31.602604 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:32.313791 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:32.313756 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:32.572117 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:32.572030 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" event={"ID":"0d2361f9-9279-41de-b218-d4d64d23699c","Type":"ContainerStarted","Data":"02f9e1684f284552c8df6474ba3860fe20070d164672d3e938e41ae19efeeb55"} Jun 05 13:10:32.572117 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:32.572093 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" event={"ID":"0d2361f9-9279-41de-b218-d4d64d23699c","Type":"ContainerStarted","Data":"f4fcd00f1ac894915c67d8b4776b9d6ca96426ce0b2e9549fbc6c723f6bd8ea9"} Jun 05 13:10:32.612668 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:32.612623 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-ggbb7" podStartSLOduration=1.419717786 podStartE2EDuration="2.612608971s" totalImagesPullingTime="1.192891185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:32.611498501 +0000 UTC m=+245.252822302" watchObservedRunningTime="2026-06-05 13:10:32.612608971 +0000 UTC m=+245.253932771" Jun 05 13:10:34.420073 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:34.420041 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:34.854168 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:34.854136 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:35.123173 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.123098 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr"] Jun 05 13:10:35.126924 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.126906 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.143237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.143212 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 05 13:10:35.144434 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.144413 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 05 13:10:35.144545 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.144449 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5kqql\"" Jun 05 13:10:35.144545 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.144524 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 05 13:10:35.146511 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.146488 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-grznq"] Jun 05 13:10:35.149900 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.149883 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.154019 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.154003 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 05 13:10:35.154149 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.154126 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dsclp\"" Jun 05 13:10:35.154710 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.154695 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 05 13:10:35.155178 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.155165 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 05 13:10:35.160587 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.160569 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr"] Jun 05 13:10:35.173991 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.173942 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:35.174923 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.174906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-wtmp\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175022 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.174943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-tls\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175022 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175008 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4wcb\" (UniqueName: \"kubernetes.io/projected/73530901-1d93-43e1-b18e-99ee84a5e052-kube-api-access-f4wcb\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.175098 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175032 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-textfile\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175098 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/73530901-1d93-43e1-b18e-99ee84a5e052-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.175190 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175101 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03399703-6846-44f2-856c-614e1df5e51c-metrics-client-ca\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175190 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175138 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175190 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-sys\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.175356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73530901-1d93-43e1-b18e-99ee84a5e052-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.175356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.175356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175331 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-root\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175379 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-accelerators-collector-config\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxmj4\" (UniqueName: \"kubernetes.io/projected/03399703-6846-44f2-856c-614e1df5e51c-kube-api-access-rxmj4\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.175533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.175459 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.276764 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.276925 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276787 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-wtmp\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.276925 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276831 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-tls\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.276925 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4wcb\" (UniqueName: \"kubernetes.io/projected/73530901-1d93-43e1-b18e-99ee84a5e052-kube-api-access-f4wcb\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.276925 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276898 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-textfile\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.276925 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276923 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/73530901-1d93-43e1-b18e-99ee84a5e052-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.277134 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.276963 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03399703-6846-44f2-856c-614e1df5e51c-metrics-client-ca\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.277835 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.277462 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-wtmp\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.277835 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.277810 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-textfile\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.277849 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/73530901-1d93-43e1-b18e-99ee84a5e052-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.278007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.277035 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.277950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-sys\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.278195 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73530901-1d93-43e1-b18e-99ee84a5e052-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.278195 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278097 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.278195 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-root\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278195 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278161 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03399703-6846-44f2-856c-614e1df5e51c-metrics-client-ca\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278404 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278232 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-sys\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278710 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278686 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-accelerators-collector-config\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278854 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-accelerators-collector-config\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.278970 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.278956 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxmj4\" (UniqueName: \"kubernetes.io/projected/03399703-6846-44f2-856c-614e1df5e51c-kube-api-access-rxmj4\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.280737 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.280711 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-tls\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.280828 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.280789 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03399703-6846-44f2-856c-614e1df5e51c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.280828 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.280816 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03399703-6846-44f2-856c-614e1df5e51c-root\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.280966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.280900 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.284292 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.281430 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73530901-1d93-43e1-b18e-99ee84a5e052-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.284292 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.282075 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.285395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.285371 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/73530901-1d93-43e1-b18e-99ee84a5e052-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.298216 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.298190 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4wcb\" (UniqueName: \"kubernetes.io/projected/73530901-1d93-43e1-b18e-99ee84a5e052-kube-api-access-f4wcb\") pod \"kube-state-metrics-57bbf8bfb5-28lpr\" (UID: \"73530901-1d93-43e1-b18e-99ee84a5e052\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.314065 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.314037 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxmj4\" (UniqueName: \"kubernetes.io/projected/03399703-6846-44f2-856c-614e1df5e51c-kube-api-access-rxmj4\") pod \"node-exporter-grznq\" (UID: \"03399703-6846-44f2-856c-614e1df5e51c\") " pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.404310 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.404186 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:35.436983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.436949 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" Jun 05 13:10:35.460320 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.460287 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-grznq" Jun 05 13:10:35.473575 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:35.473547 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03399703_6846_44f2_856c_614e1df5e51c.slice/crio-aa8867dab019867c4a41943313656f51f27cb3c013a192783271652f6483b7b5 WatchSource:0}: Error finding container aa8867dab019867c4a41943313656f51f27cb3c013a192783271652f6483b7b5: Status 404 returned error can't find the container with id aa8867dab019867c4a41943313656f51f27cb3c013a192783271652f6483b7b5 Jun 05 13:10:35.584774 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.584738 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grznq" event={"ID":"03399703-6846-44f2-856c-614e1df5e51c","Type":"ContainerStarted","Data":"aa8867dab019867c4a41943313656f51f27cb3c013a192783271652f6483b7b5"} Jun 05 13:10:35.632266 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.632220 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr"] Jun 05 13:10:35.635061 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:35.635033 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73530901_1d93_43e1_b18e_99ee84a5e052.slice/crio-23abd1dcfad85d477b086f2bfc093204db3ccefc7a17adabfaa787894fdc4d54 WatchSource:0}: Error finding container 23abd1dcfad85d477b086f2bfc093204db3ccefc7a17adabfaa787894fdc4d54: Status 404 returned error can't find the container with id 23abd1dcfad85d477b086f2bfc093204db3ccefc7a17adabfaa787894fdc4d54 Jun 05 13:10:35.686886 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.686799 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:35.985333 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:35.985299 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:36.399374 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:36.399282 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:36.590697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:36.590661 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" event={"ID":"73530901-1d93-43e1-b18e-99ee84a5e052","Type":"ContainerStarted","Data":"23abd1dcfad85d477b086f2bfc093204db3ccefc7a17adabfaa787894fdc4d54"} Jun 05 13:10:36.592599 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:36.592530 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grznq" event={"ID":"03399703-6846-44f2-856c-614e1df5e51c","Type":"ContainerStarted","Data":"2e87364292a6b84d4e9304d4afd007b0e156aefe70ffa49b62a9e45af92af12a"} Jun 05 13:10:36.742744 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:36.742710 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:36.974014 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:36.973986 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:37.598570 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:37.598533 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" event={"ID":"73530901-1d93-43e1-b18e-99ee84a5e052","Type":"ContainerStarted","Data":"b4ebf5b3cf86faa122d8c960e006a5bdce60249e5c59f584c3accbf97ad3a798"} Jun 05 13:10:37.599006 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:37.598597 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" event={"ID":"73530901-1d93-43e1-b18e-99ee84a5e052","Type":"ContainerStarted","Data":"da24d3c6ff49a904136ca8087d7a7218b9462b76433c9d0f1ded6324e84e3b89"} Jun 05 13:10:37.599006 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:37.598623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" event={"ID":"73530901-1d93-43e1-b18e-99ee84a5e052","Type":"ContainerStarted","Data":"698a75c6b7c99f489056424811b971529ffe210205d3e241a8a4866bfe920c1d"} Jun 05 13:10:37.600302 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:37.600279 2570 generic.go:356] "Generic (PLEG): container finished" podID="03399703-6846-44f2-856c-614e1df5e51c" containerID="2e87364292a6b84d4e9304d4afd007b0e156aefe70ffa49b62a9e45af92af12a" exitCode=0 Jun 05 13:10:37.600385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:37.600334 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grznq" event={"ID":"03399703-6846-44f2-856c-614e1df5e51c","Type":"ContainerDied","Data":"2e87364292a6b84d4e9304d4afd007b0e156aefe70ffa49b62a9e45af92af12a"} Jun 05 13:10:37.656817 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:37.656755 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-28lpr" podStartSLOduration=1.323473162 podStartE2EDuration="2.656733618s" totalImagesPullingTime="1.333260456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:37.65538571 +0000 UTC m=+250.296709511" watchObservedRunningTime="2026-06-05 13:10:37.656733618 +0000 UTC m=+250.298057418" Jun 05 13:10:38.606921 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.606849 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grznq" event={"ID":"03399703-6846-44f2-856c-614e1df5e51c","Type":"ContainerStarted","Data":"1fbb204b3c30cc7128ef84f4351626662f8630bf6e53271094eb5fb807c40099"} Jun 05 13:10:38.607374 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.606979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-grznq" event={"ID":"03399703-6846-44f2-856c-614e1df5e51c","Type":"ContainerStarted","Data":"1c26484213396bdd7b1c2c32d297b52073e95800e86fab8fe05ff64a06ca8f82"} Jun 05 13:10:38.610660 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.610632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:10:38.613007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.612984 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2-metrics-certs\") pod \"network-metrics-daemon-8d2pt\" (UID: \"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2\") " pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:10:38.623886 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.623864 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:38.848922 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.848891 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:38.881143 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.881069 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cph74\"" Jun 05 13:10:38.887719 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:38.887703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8d2pt" Jun 05 13:10:39.018213 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.018185 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:39.041485 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.041431 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-grznq" podStartSLOduration=3.021177853 podStartE2EDuration="4.041408847s" totalImagesPullingTime="951.543657ms" totalInitContainerRuntime="68.687337ms" isStatefulPod=true podCreationTimestamp="2026-06-05 13:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:38.656049665 +0000 UTC m=+251.297373468" watchObservedRunningTime="2026-06-05 13:10:39.041408847 +0000 UTC m=+251.682732643" Jun 05 13:10:39.043526 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.043499 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8d2pt"] Jun 05 13:10:39.045765 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:39.045739 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2283fa84_7fcd_4a8c_9cf5_ee90f6478fc2.slice/crio-f552d199a1f739bd6ad794adb4b82e7ce9a3c8778bcf431111282e65b8342fb4 WatchSource:0}: Error finding container f552d199a1f739bd6ad794adb4b82e7ce9a3c8778bcf431111282e65b8342fb4: Status 404 returned error can't find the container with id f552d199a1f739bd6ad794adb4b82e7ce9a3c8778bcf431111282e65b8342fb4 Jun 05 13:10:39.538323 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.538295 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s"] Jun 05 13:10:39.546745 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.546717 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:39.548551 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.548501 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:39.557010 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.556075 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-twm8m\"" Jun 05 13:10:39.557010 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.556346 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 05 13:10:39.612216 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.612150 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8d2pt" event={"ID":"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2","Type":"ContainerStarted","Data":"f552d199a1f739bd6ad794adb4b82e7ce9a3c8778bcf431111282e65b8342fb4"} Jun 05 13:10:39.620000 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.619972 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcb67\" (UniqueName: \"kubernetes.io/projected/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-kube-api-access-fcb67\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:39.620159 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.620015 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:39.627459 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.627431 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s"] Jun 05 13:10:39.721235 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.720861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcb67\" (UniqueName: \"kubernetes.io/projected/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-kube-api-access-fcb67\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:39.721235 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.720909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:39.721235 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:39.721069 2570 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 05 13:10:39.721235 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:10:39.721149 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-monitoring-plugin-cert podName:8fe1fbde-e3fb-4c4b-b359-387de7f15ac9 nodeName:}" failed. No retries permitted until 2026-06-05 13:10:40.221128485 +0000 UTC m=+252.862452279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-nkb9s" (UID: "8fe1fbde-e3fb-4c4b-b359-387de7f15ac9") : secret "monitoring-plugin-cert" not found Jun 05 13:10:39.768748 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.768218 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-65cd568d69-nzcpg"] Jun 05 13:10:39.768748 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.768711 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcb67\" (UniqueName: \"kubernetes.io/projected/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-kube-api-access-fcb67\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:39.772812 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.772784 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.787059 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.786712 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 05 13:10:39.787059 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.786960 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 05 13:10:39.787304 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.787211 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 05 13:10:39.787372 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.787344 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 05 13:10:39.787511 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.787493 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2b0e2bulq9f82\"" Jun 05 13:10:39.792366 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.792313 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dls9m\"" Jun 05 13:10:39.821674 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.821623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9glz\" (UniqueName: \"kubernetes.io/projected/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-kube-api-access-s9glz\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.821846 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.821756 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.821846 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.821811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-secret-metrics-server-client-certs\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.821934 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.821895 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-secret-metrics-server-tls\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.821985 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.821944 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-client-ca-bundle\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.822036 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.822007 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-metrics-server-audit-profiles\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.822087 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.822043 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-audit-log\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.837411 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.837379 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65cd568d69-nzcpg"] Jun 05 13:10:39.922962 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.922925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9glz\" (UniqueName: \"kubernetes.io/projected/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-kube-api-access-s9glz\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923121 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.922993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923121 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923021 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-secret-metrics-server-client-certs\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923121 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923054 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-secret-metrics-server-tls\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923278 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-client-ca-bundle\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923336 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923309 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-metrics-server-audit-profiles\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923383 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923347 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-audit-log\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923730 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923706 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-audit-log\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.923828 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.923774 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.924353 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.924330 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-metrics-server-audit-profiles\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.925748 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.925719 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-secret-metrics-server-tls\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.925914 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.925894 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-client-ca-bundle\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.925988 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.925966 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-secret-metrics-server-client-certs\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:39.926879 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.926856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:39.937072 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:39.937053 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9glz\" (UniqueName: \"kubernetes.io/projected/4e7b7b3a-c15b-40e8-99b5-ed03042928e4-kube-api-access-s9glz\") pod \"metrics-server-65cd568d69-nzcpg\" (UID: \"4e7b7b3a-c15b-40e8-99b5-ed03042928e4\") " pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:40.085969 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.085946 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:10:40.213052 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.213016 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:40.226553 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.224783 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:40.227775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.227752 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8fe1fbde-e3fb-4c4b-b359-387de7f15ac9-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-nkb9s\" (UID: \"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:40.314075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.314051 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65cd568d69-nzcpg"] Jun 05 13:10:40.316091 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:40.316039 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7b7b3a_c15b_40e8_99b5_ed03042928e4.slice/crio-59527b1101fe36e77d49fd9b511067588c85da219887cd7537833261cb7b1d1c WatchSource:0}: Error finding container 59527b1101fe36e77d49fd9b511067588c85da219887cd7537833261cb7b1d1c: Status 404 returned error can't find the container with id 59527b1101fe36e77d49fd9b511067588c85da219887cd7537833261cb7b1d1c Jun 05 13:10:40.459740 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.459703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:40.559051 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.559022 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c6897df99-7x5ng"] Jun 05 13:10:40.563431 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.563412 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.571187 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.571140 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 05 13:10:40.571187 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.571162 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 05 13:10:40.571442 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.571162 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 05 13:10:40.571442 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.571339 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mgjnm\"" Jun 05 13:10:40.574607 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.574593 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 05 13:10:40.574951 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.574936 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 05 13:10:40.580699 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.580682 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 05 13:10:40.605562 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.605536 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c6897df99-7x5ng"] Jun 05 13:10:40.621058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.621022 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" event={"ID":"4e7b7b3a-c15b-40e8-99b5-ed03042928e4","Type":"ContainerStarted","Data":"59527b1101fe36e77d49fd9b511067588c85da219887cd7537833261cb7b1d1c"} Jun 05 13:10:40.622803 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.622776 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8d2pt" event={"ID":"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2","Type":"ContainerStarted","Data":"469be9c9d0cea9db9e534a4273ca821004639b1366d3d22717deb90fba6bdcdf"} Jun 05 13:10:40.622896 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.622816 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8d2pt" event={"ID":"2283fa84-7fcd-4a8c-9cf5-ee90f6478fc2","Type":"ContainerStarted","Data":"f08ec1bc4aff2b68dec223f0938ad1f551bfffed50cfe6b8f2efc60995a4a933"} Jun 05 13:10:40.629311 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629291 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgsq\" (UniqueName: \"kubernetes.io/projected/c2170061-4cfd-471e-8b0e-89282819502f-kube-api-access-xwgsq\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629361 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629336 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629413 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629401 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-telemeter-client-tls\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629448 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-serving-certs-ca-bundle\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629481 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629452 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-secret-telemeter-client\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629481 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629473 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-federate-client-tls\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629539 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629516 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-metrics-client-ca\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.629569 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.629555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.658001 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.657971 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s"] Jun 05 13:10:40.661902 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:40.661873 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe1fbde_e3fb_4c4b_b359_387de7f15ac9.slice/crio-cb9a0a6ca147b7b13de56c235b8098b6b446df349b12a2af2928539a687e4576 WatchSource:0}: Error finding container cb9a0a6ca147b7b13de56c235b8098b6b446df349b12a2af2928539a687e4576: Status 404 returned error can't find the container with id cb9a0a6ca147b7b13de56c235b8098b6b446df349b12a2af2928539a687e4576 Jun 05 13:10:40.713761 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.713706 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8d2pt" podStartSLOduration=252.828402864 podStartE2EDuration="4m13.713687202s" totalImagesPullingTime="885.284338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:40.713651033 +0000 UTC m=+253.354974833" watchObservedRunningTime="2026-06-05 13:10:40.713687202 +0000 UTC m=+253.355011002" Jun 05 13:10:40.729924 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.729892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-federate-client-tls\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730073 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.729935 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-metrics-client-ca\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730073 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.729958 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730157 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgsq\" (UniqueName: \"kubernetes.io/projected/c2170061-4cfd-471e-8b0e-89282819502f-kube-api-access-xwgsq\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730206 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730288 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730221 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-telemeter-client-tls\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730342 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730286 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-serving-certs-ca-bundle\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.730342 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730321 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-secret-telemeter-client\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.731040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730733 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-metrics-client-ca\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.731040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.730868 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.731213 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.731096 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2170061-4cfd-471e-8b0e-89282819502f-serving-certs-ca-bundle\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.732823 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.732790 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-telemeter-client-tls\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.732907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.732873 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-federate-client-tls\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.733118 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.733102 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-secret-telemeter-client\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.733155 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.733122 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2170061-4cfd-471e-8b0e-89282819502f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.755976 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.755946 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgsq\" (UniqueName: \"kubernetes.io/projected/c2170061-4cfd-471e-8b0e-89282819502f-kube-api-access-xwgsq\") pod \"telemeter-client-6c6897df99-7x5ng\" (UID: \"c2170061-4cfd-471e-8b0e-89282819502f\") " pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:40.873154 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:40.873077 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" Jun 05 13:10:41.031203 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.031167 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:41.076441 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:41.076384 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2170061_4cfd_471e_8b0e_89282819502f.slice/crio-464bafe4c2446cff619942956fd0fbba5d24703627062e8d5c5a20f7b897bf91 WatchSource:0}: Error finding container 464bafe4c2446cff619942956fd0fbba5d24703627062e8d5c5a20f7b897bf91: Status 404 returned error can't find the container with id 464bafe4c2446cff619942956fd0fbba5d24703627062e8d5c5a20f7b897bf91 Jun 05 13:10:41.079982 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.079953 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c6897df99-7x5ng"] Jun 05 13:10:41.221834 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.221756 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:41.460280 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.460226 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:41.628815 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.628763 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" event={"ID":"c2170061-4cfd-471e-8b0e-89282819502f","Type":"ContainerStarted","Data":"464bafe4c2446cff619942956fd0fbba5d24703627062e8d5c5a20f7b897bf91"} Jun 05 13:10:41.630040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.630014 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" event={"ID":"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9","Type":"ContainerStarted","Data":"cb9a0a6ca147b7b13de56c235b8098b6b446df349b12a2af2928539a687e4576"} Jun 05 13:10:41.679976 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:41.679937 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:42.113042 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.113002 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:10:42.119807 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.119775 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.127445 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.127388 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 05 13:10:42.127928 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.127740 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 05 13:10:42.132181 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.132062 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 05 13:10:42.132454 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.132361 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 05 13:10:42.133054 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.132626 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 05 13:10:42.133054 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.132697 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 05 13:10:42.133054 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.132853 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 05 13:10:42.136185 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.133154 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 05 13:10:42.136382 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.136264 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-chbjt\"" Jun 05 13:10:42.141288 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.140970 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 05 13:10:42.141288 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.141199 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 05 13:10:42.141288 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.141209 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 05 13:10:42.141495 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.141453 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 05 13:10:42.147317 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147294 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-web-config\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147401 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147342 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147401 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147377 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147524 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147464 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147579 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147519 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147579 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147671 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147671 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147622 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147671 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147647 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147806 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147678 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147806 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147702 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147806 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147745 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147806 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147782 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-config\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147809 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147841 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147902 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr6n7\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-kube-api-access-fr6n7\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147954 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.147984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.147976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-config-out\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.148224 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.148000 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.179863 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.179794 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:42.184949 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.184918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cp24o9ojus79l\"" Jun 05 13:10:42.186665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.186635 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 05 13:10:42.189505 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.189478 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:10:42.191342 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.191319 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 05 13:10:42.248795 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248760 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.248980 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248803 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-config-out\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.248980 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.248980 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-web-config\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.248980 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248896 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.248980 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.248966 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249060 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249104 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249129 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249153 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249183 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249212 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249331 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-config\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249430 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249630 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr6n7\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-kube-api-access-fr6n7\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.249875 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.249731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.250513 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.250412 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.250627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.250527 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.251330 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.250997 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.251419 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.251393 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.252607 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.252581 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.252981 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.252957 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.253522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.253497 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.254597 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.254572 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.255398 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.255295 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-web-config\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.256706 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.256204 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.256706 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.256635 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.257376 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.256754 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-config-out\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.258121 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.258074 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.258632 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.258584 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.258843 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.258806 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.259156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.259137 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.259648 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.259603 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.260891 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.260870 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-config\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.288832 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.288753 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr6n7\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-kube-api-access-fr6n7\") pod \"prometheus-k8s-0\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.437358 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.437232 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:42.637683 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.637623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" event={"ID":"4e7b7b3a-c15b-40e8-99b5-ed03042928e4","Type":"ContainerStarted","Data":"f5cd07380cbfcf159777dd410c016ac57ae2ca1530b88b3a610a7e1eca457add"} Jun 05 13:10:42.710191 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.710087 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" podStartSLOduration=2.289228104 podStartE2EDuration="3.710069605s" totalImagesPullingTime="1.420841501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:42.709390974 +0000 UTC m=+255.350714797" watchObservedRunningTime="2026-06-05 13:10:42.710069605 +0000 UTC m=+255.351393405" Jun 05 13:10:42.757433 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.757405 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:10:42.760902 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:10:42.760878 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda542702e_8139_4eac_bf3b_9315b6a60a05.slice/crio-9dc79346acce2aa985cd936c028b1803c05b5dc4f1f2f06e4707f4de0c707563 WatchSource:0}: Error finding container 9dc79346acce2aa985cd936c028b1803c05b5dc4f1f2f06e4707f4de0c707563: Status 404 returned error can't find the container with id 9dc79346acce2aa985cd936c028b1803c05b5dc4f1f2f06e4707f4de0c707563 Jun 05 13:10:42.777465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:42.777443 2570 scope.go:117] "RemoveContainer" containerID="602788f06ea4cad55d2484709f3c92c7f2826f220cefe08dcbe01ebe066ff39c" Jun 05 13:10:43.644867 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.644779 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:10:43.645324 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.644893 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" event={"ID":"75838c9e-e3ab-4c39-848c-528d0dcf1fac","Type":"ContainerStarted","Data":"629296f52cdc6e6d0210c8de618617620aafebbb01f23cf7bf460792fe4556b9"} Jun 05 13:10:43.645506 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.645344 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:10:43.647773 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.647677 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" event={"ID":"c2170061-4cfd-471e-8b0e-89282819502f","Type":"ContainerStarted","Data":"d4a397c5b089584faf8bacc21a793fb2e574d85b7b9e2d14a1f2b2a3fd5379f3"} Jun 05 13:10:43.649775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.649732 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"9dc79346acce2aa985cd936c028b1803c05b5dc4f1f2f06e4707f4de0c707563"} Jun 05 13:10:43.652263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.651910 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" event={"ID":"8fe1fbde-e3fb-4c4b-b359-387de7f15ac9","Type":"ContainerStarted","Data":"a52587e18f0d3ae977de80bd9315486f88cf92ed4786ba6083a7f5b3118063c0"} Jun 05 13:10:43.652263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.652004 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" Jun 05 13:10:43.652263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.652041 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:43.658635 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.658611 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" Jun 05 13:10:43.723559 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.723499 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-jjn5c" podStartSLOduration=53.29477683 podStartE2EDuration="56.723477904s" totalImagesPullingTime="3.428701074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:43.722021326 +0000 UTC m=+256.363345128" watchObservedRunningTime="2026-06-05 13:10:43.723477904 +0000 UTC m=+256.364801706" Jun 05 13:10:43.877887 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:43.877696 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-nkb9s" podStartSLOduration=2.955400868 podStartE2EDuration="4.877674827s" totalImagesPullingTime="1.922273959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:43.792233518 +0000 UTC m=+256.433557321" watchObservedRunningTime="2026-06-05 13:10:43.877674827 +0000 UTC m=+256.518998631" Jun 05 13:10:44.095544 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.095511 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:44.533530 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.533479 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:44.657898 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.657864 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" event={"ID":"c2170061-4cfd-471e-8b0e-89282819502f","Type":"ContainerStarted","Data":"0ecf3c5c1068a8f17b2446608109b61a4d1d7ef4c069b368a1a024cafd98655f"} Jun 05 13:10:44.658360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.657922 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" event={"ID":"c2170061-4cfd-471e-8b0e-89282819502f","Type":"ContainerStarted","Data":"89497c4e44ec0d1354db50f9733b761fdce1bf07c86723c846dd75d6d6abaafb"} Jun 05 13:10:44.659368 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.659336 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac" exitCode=0 Jun 05 13:10:44.659482 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.659419 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac"} Jun 05 13:10:44.702355 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.702305 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c6897df99-7x5ng" podStartSLOduration=1.718213788 podStartE2EDuration="4.702289409s" totalImagesPullingTime="2.984075621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:44.695519689 +0000 UTC m=+257.336843502" watchObservedRunningTime="2026-06-05 13:10:44.702289409 +0000 UTC m=+257.343613248" Jun 05 13:10:44.858632 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:44.858545 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:45.387277 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:45.387220 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:45.562657 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:45.562617 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:46.458235 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:46.458203 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:46.800559 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:46.800527 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:48.679769 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:48.679693 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e"} Jun 05 13:10:48.679769 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:48.679756 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927"} Jun 05 13:10:50.693661 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:50.693575 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1"} Jun 05 13:10:50.693661 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:50.693639 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e"} Jun 05 13:10:50.694082 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:50.693668 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e"} Jun 05 13:10:50.694082 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:50.693692 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerStarted","Data":"b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9"} Jun 05 13:10:52.437658 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:52.437622 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:10:52.810770 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:52.810740 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:53.030262 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:53.030212 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:53.996590 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:53.996550 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:54.224584 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:54.224553 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:58.996651 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:58.996617 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:10:59.139806 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:10:59.139753 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:00.086843 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:00.086804 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:11:00.087223 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:00.086865 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:11:02.012306 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:02.012266 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:02.201978 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:02.201946 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:07.294062 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:11:07.294018 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t27mz" podUID="a6d4bb4b-66ac-4457-99e4-1acafbf1d98e" Jun 05 13:11:07.294062 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:11:07.294042 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" Jun 05 13:11:07.761640 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:07.761612 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:11:07.761640 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:07.761643 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t27mz" Jun 05 13:11:08.375260 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:08.375187 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:08.528883 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:08.528843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:11.229902 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.229867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:11:11.229902 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.229904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:11:11.232207 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.232180 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6d4bb4b-66ac-4457-99e4-1acafbf1d98e-metrics-tls\") pod \"dns-default-t27mz\" (UID: \"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e\") " pod="openshift-dns/dns-default-t27mz" Jun 05 13:11:11.232471 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.232448 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"image-registry-6f876c6468-qmzf4\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:11:11.374940 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.374911 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m678x\"" Jun 05 13:11:11.375087 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.374944 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d25hl\"" Jun 05 13:11:11.382931 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.382915 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:11:11.383014 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.382999 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t27mz" Jun 05 13:11:11.551306 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.551088 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=22.80448123 podStartE2EDuration="29.551065284s" totalImagesPullingTime="6.726049466s" totalInitContainerRuntime="20.534588ms" isStatefulPod=false podCreationTimestamp="2026-06-05 13:10:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:10:50.769899335 +0000 UTC m=+263.411223146" watchObservedRunningTime="2026-06-05 13:11:11.551065284 +0000 UTC m=+284.192389087" Jun 05 13:11:11.552474 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.552456 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t27mz"] Jun 05 13:11:11.554024 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:11:11.554000 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d4bb4b_66ac_4457_99e4_1acafbf1d98e.slice/crio-a3e12ae8d7a4d08498cd201fee73be4b0117c122e17b32b5169521ffc6a1dd34 WatchSource:0}: Error finding container a3e12ae8d7a4d08498cd201fee73be4b0117c122e17b32b5169521ffc6a1dd34: Status 404 returned error can't find the container with id a3e12ae8d7a4d08498cd201fee73be4b0117c122e17b32b5169521ffc6a1dd34 Jun 05 13:11:11.567777 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.567748 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f876c6468-qmzf4"] Jun 05 13:11:11.570399 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:11:11.570378 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c5754e_c750_4fdf_a2e5_19a92fa6671e.slice/crio-bdb6bb1c6736b8648303572a5989dea635ac4440c032cbe5f80228441bec0403 WatchSource:0}: Error finding container bdb6bb1c6736b8648303572a5989dea635ac4440c032cbe5f80228441bec0403: Status 404 returned error can't find the container with id bdb6bb1c6736b8648303572a5989dea635ac4440c032cbe5f80228441bec0403 Jun 05 13:11:11.781498 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.781423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" event={"ID":"64c5754e-c750-4fdf-a2e5-19a92fa6671e","Type":"ContainerStarted","Data":"8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1"} Jun 05 13:11:11.781498 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.781466 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" event={"ID":"64c5754e-c750-4fdf-a2e5-19a92fa6671e","Type":"ContainerStarted","Data":"bdb6bb1c6736b8648303572a5989dea635ac4440c032cbe5f80228441bec0403"} Jun 05 13:11:11.781498 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.781483 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:11:11.781498 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.781495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t27mz" event={"ID":"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e","Type":"ContainerStarted","Data":"a3e12ae8d7a4d08498cd201fee73be4b0117c122e17b32b5169521ffc6a1dd34"} Jun 05 13:11:11.817611 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:11.817561 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" podStartSLOduration=284.817545763 podStartE2EDuration="4m44.817545763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:11:11.817107375 +0000 UTC m=+284.458431176" watchObservedRunningTime="2026-06-05 13:11:11.817545763 +0000 UTC m=+284.458869563" Jun 05 13:11:12.786230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:12.786194 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1a06d16-edfd-48d5-9168-0cd21d991511" containerID="736b719f97e1b822efc38418ccff47cfcd0afba4d9c271a04b0734a0e2613696" exitCode=0 Jun 05 13:11:12.786676 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:12.786272 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" event={"ID":"e1a06d16-edfd-48d5-9168-0cd21d991511","Type":"ContainerDied","Data":"736b719f97e1b822efc38418ccff47cfcd0afba4d9c271a04b0734a0e2613696"} Jun 05 13:11:12.786724 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:12.786707 2570 scope.go:117] "RemoveContainer" containerID="736b719f97e1b822efc38418ccff47cfcd0afba4d9c271a04b0734a0e2613696" Jun 05 13:11:13.796638 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:13.796597 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-mkvx6" event={"ID":"e1a06d16-edfd-48d5-9168-0cd21d991511","Type":"ContainerStarted","Data":"8b6449ab73f8b71bf86b2e453ca8d4cbe7c6285dc812cab0159ccd2a4e9bb83b"} Jun 05 13:11:14.802563 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:14.802526 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t27mz" event={"ID":"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e","Type":"ContainerStarted","Data":"e3dc91492f54aa1e3209fe0036627d99b303fec8ed942bfff2879454cebe9a8b"} Jun 05 13:11:14.802943 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:14.802606 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t27mz" event={"ID":"a6d4bb4b-66ac-4457-99e4-1acafbf1d98e","Type":"ContainerStarted","Data":"ea8a8168eff481680b616a82c73f59dccc413569d325b36ea88521c4e6c3f5d5"} Jun 05 13:11:14.802943 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:14.802627 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t27mz" Jun 05 13:11:14.839603 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:14.839558 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t27mz" podStartSLOduration=251.700905515 podStartE2EDuration="4m13.83954309s" totalImagesPullingTime="2.138637575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:11:14.838664538 +0000 UTC m=+287.479988340" watchObservedRunningTime="2026-06-05 13:11:14.83954309 +0000 UTC m=+287.480866891" Jun 05 13:11:15.779088 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:15.779024 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:15.940926 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:15.940889 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:20.093772 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:20.093740 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:11:20.097505 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:20.097481 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-65cd568d69-nzcpg" Jun 05 13:11:22.839132 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:22.839043 2570 generic.go:356] "Generic (PLEG): container finished" podID="ee227f79-486b-42c0-9500-cbdb8540900c" containerID="314a144230fac56370430773231e7c6c86c0944a47385f34797479179088c3a3" exitCode=0 Jun 05 13:11:22.839132 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:22.839119 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-tsj2d" event={"ID":"ee227f79-486b-42c0-9500-cbdb8540900c","Type":"ContainerDied","Data":"314a144230fac56370430773231e7c6c86c0944a47385f34797479179088c3a3"} Jun 05 13:11:22.839565 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:22.839474 2570 scope.go:117] "RemoveContainer" containerID="314a144230fac56370430773231e7c6c86c0944a47385f34797479179088c3a3" Jun 05 13:11:23.845241 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:23.845204 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-tsj2d" event={"ID":"ee227f79-486b-42c0-9500-cbdb8540900c","Type":"ContainerStarted","Data":"d3c69f2435f13ba145db2093ef5c74058cfbd7fe766691e796917d09fa215406"} Jun 05 13:11:24.809493 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:24.809462 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t27mz" Jun 05 13:11:25.402267 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:25.402212 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:27.046162 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.046133 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-q2vgx_295dbce6-3dc9-42bc-b917-8c6f21a339eb/cluster-monitoring-operator/0.log" Jun 05 13:11:27.234508 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.234477 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-28lpr_73530901-1d93-43e1-b18e-99ee84a5e052/kube-state-metrics/0.log" Jun 05 13:11:27.442440 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.442366 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-28lpr_73530901-1d93-43e1-b18e-99ee84a5e052/kube-rbac-proxy-main/0.log" Jun 05 13:11:27.639407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.639377 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-28lpr_73530901-1d93-43e1-b18e-99ee84a5e052/kube-rbac-proxy-self/0.log" Jun 05 13:11:27.736551 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.736523 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:11:27.736861 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.736647 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:11:27.748467 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.748447 2570 kubelet.go:1648] "Image garbage collection succeeded" Jun 05 13:11:27.845926 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:27.845777 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65cd568d69-nzcpg_4e7b7b3a-c15b-40e8-99b5-ed03042928e4/metrics-server/0.log" Jun 05 13:11:28.037048 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:28.036967 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-nkb9s_8fe1fbde-e3fb-4c4b-b359-387de7f15ac9/monitoring-plugin/0.log" Jun 05 13:11:28.834385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:28.834355 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grznq_03399703-6846-44f2-856c-614e1df5e51c/init-textfile/0.log" Jun 05 13:11:29.035936 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:29.035905 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grznq_03399703-6846-44f2-856c-614e1df5e51c/node-exporter/0.log" Jun 05 13:11:29.234233 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:29.234206 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grznq_03399703-6846-44f2-856c-614e1df5e51c/kube-rbac-proxy/0.log" Jun 05 13:11:30.101227 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.101197 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:30.203273 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.203219 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:30.336950 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.336913 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:30.637159 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.637133 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/init-config-reloader/0.log" Jun 05 13:11:30.787233 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.787200 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:30.837026 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.836997 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/prometheus/0.log" Jun 05 13:11:30.881341 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:30.881311 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:31.036207 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.036183 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/config-reloader/0.log" Jun 05 13:11:31.243101 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.243068 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/thanos-sidecar/0.log" Jun 05 13:11:31.387204 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.387127 2570 patch_prober.go:28] interesting pod/image-registry-6f876c6468-qmzf4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 05 13:11:31.387370 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.387190 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 05 13:11:31.453694 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.453639 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/kube-rbac-proxy-web/0.log" Jun 05 13:11:31.640102 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.640019 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/kube-rbac-proxy/0.log" Jun 05 13:11:31.833888 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:31.833855 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a542702e-8139-4eac-bf3b-9315b6a60a05/kube-rbac-proxy-thanos/0.log" Jun 05 13:11:32.060380 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:32.060349 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-ggbb7_0d2361f9-9279-41de-b218-d4d64d23699c/prometheus-operator/0.log" Jun 05 13:11:32.267573 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:32.267546 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-ggbb7_0d2361f9-9279-41de-b218-d4d64d23699c/kube-rbac-proxy/0.log" Jun 05 13:11:32.443969 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:32.443887 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-rcgd9_4239b006-96bc-4178-a4f6-a9548fd18b91/prometheus-operator-admission-webhook/0.log" Jun 05 13:11:32.639168 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:32.639140 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c6897df99-7x5ng_c2170061-4cfd-471e-8b0e-89282819502f/telemeter-client/0.log" Jun 05 13:11:32.834785 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:32.834753 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c6897df99-7x5ng_c2170061-4cfd-471e-8b0e-89282819502f/reload/0.log" Jun 05 13:11:33.036523 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:33.036497 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c6897df99-7x5ng_c2170061-4cfd-471e-8b0e-89282819502f/kube-rbac-proxy/0.log" Jun 05 13:11:33.801407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:33.801382 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:11:33.938113 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:33.938085 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:34.194527 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:34.194439 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:34.319347 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:34.319312 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:34.461227 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:34.461156 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-lpd2g_611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac/networking-console-plugin/0.log" Jun 05 13:11:34.644529 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:34.644505 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:11:34.846179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:34.846151 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/3.log" Jun 05 13:11:36.044789 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:36.044759 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t27mz_a6d4bb4b-66ac-4457-99e4-1acafbf1d98e/dns/0.log" Jun 05 13:11:36.250859 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:36.250833 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t27mz_a6d4bb4b-66ac-4457-99e4-1acafbf1d98e/kube-rbac-proxy/0.log" Jun 05 13:11:36.851730 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:36.851704 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2d6ll_e3e1da7e-c5f3-4a04-95c8-230051ddcafe/dns-node-resolver/0.log" Jun 05 13:11:37.645232 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:37.645202 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f876c6468-qmzf4_64c5754e-c750-4fdf-a2e5-19a92fa6671e/registry/0.log" Jun 05 13:11:38.040034 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:38.040006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zg2xk_3a2c2470-f752-457d-b759-fad45632ee5f/node-ca/0.log" Jun 05 13:11:38.633075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:38.633042 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d2nvt_f5dcc392-3021-4ab6-8b13-a85ec0cfedf4/serve-healthcheck-canary/0.log" Jun 05 13:11:41.174609 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:41.174573 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:41.471701 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:41.471571 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:42.437731 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:42.437627 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:11:42.463146 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:42.463065 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:11:42.954941 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:42.954913 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:11:46.124117 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:46.124080 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:46.223804 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:46.223773 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:52.092057 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:52.092022 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:52.288405 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:52.288375 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:53.791717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:53.791682 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:53.915570 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:53.915538 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:55.035608 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:55.035574 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f876c6468-qmzf4"] Jun 05 13:11:55.344263 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:55.344174 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:55.532295 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:55.532237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:55.644149 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:55.644055 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:59.473634 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:59.473596 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:11:59.554553 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:11:59.554522 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:00.987373 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987330 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:12:00.987865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987674 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="prometheus" containerID="cri-o://35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927" gracePeriod=600 Jun 05 13:12:00.987865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987684 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-thanos" containerID="cri-o://812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1" gracePeriod=600 Jun 05 13:12:00.987865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987704 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="thanos-sidecar" containerID="cri-o://b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9" gracePeriod=600 Jun 05 13:12:00.987865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987737 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy" containerID="cri-o://249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e" gracePeriod=600 Jun 05 13:12:00.987865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987738 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="config-reloader" containerID="cri-o://8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e" gracePeriod=600 Jun 05 13:12:00.987865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:00.987716 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-web" containerID="cri-o://91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e" gracePeriod=600 Jun 05 13:12:02.008650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008615 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1" exitCode=0 Jun 05 13:12:02.008650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008640 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e" exitCode=0 Jun 05 13:12:02.008650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008647 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9" exitCode=0 Jun 05 13:12:02.008650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008652 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e" exitCode=0 Jun 05 13:12:02.008650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008658 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927" exitCode=0 Jun 05 13:12:02.009152 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008683 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1"} Jun 05 13:12:02.009152 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008746 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e"} Jun 05 13:12:02.009152 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008772 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9"} Jun 05 13:12:02.009152 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008797 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e"} Jun 05 13:12:02.009152 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.008833 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927"} Jun 05 13:12:02.237797 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.237769 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:02.278965 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.278880 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-trusted-ca-bundle\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.278965 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.278918 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-kube-rbac-proxy\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.278965 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.278938 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-tls-assets\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.278986 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-web-config\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279004 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-metrics-client-certs\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279020 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-config-out\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279039 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-tls\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279067 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-2\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279098 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-0\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279130 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-kubelet-serving-ca-bundle\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279166 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-serving-certs-ca-bundle\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279194 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-grpc-tls\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279235 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279292 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279338 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-thanos-prometheus-http-client-file\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279365 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-1\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279405 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-db\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279435 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr6n7\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-kube-api-access-fr6n7\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279434 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279469 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-metrics-client-ca\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.279691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279507 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-config\") pod \"a542702e-8139-4eac-bf3b-9315b6a60a05\" (UID: \"a542702e-8139-4eac-bf3b-9315b6a60a05\") " Jun 05 13:12:02.280138 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279776 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.280138 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.279790 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.282225 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.281975 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.282369 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.282235 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-config" (OuterVolumeSpecName: "config") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.282725 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.282696 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.283101 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283043 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.283222 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283199 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.283575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283383 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:02.283575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283445 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.283575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283480 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:12:02.283575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283508 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.283575 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283549 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.283906 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.283798 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 05 13:12:02.284600 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.284538 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-config-out" (OuterVolumeSpecName: "config-out") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 05 13:12:02.284600 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.284563 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.285104 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.285067 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-kube-api-access-fr6n7" (OuterVolumeSpecName: "kube-api-access-fr6n7") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "kube-api-access-fr6n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:12:02.285104 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.285097 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.285710 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.285664 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.285868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.285840 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.296911 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.296879 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-web-config" (OuterVolumeSpecName: "web-config") pod "a542702e-8139-4eac-bf3b-9315b6a60a05" (UID: "a542702e-8139-4eac-bf3b-9315b6a60a05"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:02.380478 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380436 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380478 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380472 2570 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-grpc-tls\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380486 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380501 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380513 2570 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380526 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380539 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-db\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380550 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr6n7\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-kube-api-access-fr6n7\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380562 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-metrics-client-ca\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380574 2570 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-config\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380584 2570 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-kube-rbac-proxy\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380597 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a542702e-8139-4eac-bf3b-9315b6a60a05-tls-assets\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380609 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-web-config\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380621 2570 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-metrics-client-certs\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380633 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a542702e-8139-4eac-bf3b-9315b6a60a05-config-out\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380644 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a542702e-8139-4eac-bf3b-9315b6a60a05-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380660 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380676 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.380697 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.380691 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a542702e-8139-4eac-bf3b-9315b6a60a05-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:02.944641 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:02.944603 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:03.015582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.015549 2570 generic.go:356] "Generic (PLEG): container finished" podID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerID="91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e" exitCode=0 Jun 05 13:12:03.015958 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.015629 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.015958 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.015638 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e"} Jun 05 13:12:03.015958 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.015708 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a542702e-8139-4eac-bf3b-9315b6a60a05","Type":"ContainerDied","Data":"9dc79346acce2aa985cd936c028b1803c05b5dc4f1f2f06e4707f4de0c707563"} Jun 05 13:12:03.015958 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.015742 2570 scope.go:117] "RemoveContainer" containerID="812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1" Jun 05 13:12:03.024210 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.024192 2570 scope.go:117] "RemoveContainer" containerID="249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e" Jun 05 13:12:03.031285 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.031263 2570 scope.go:117] "RemoveContainer" containerID="91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e" Jun 05 13:12:03.038524 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.038506 2570 scope.go:117] "RemoveContainer" containerID="b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9" Jun 05 13:12:03.045136 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.045113 2570 scope.go:117] "RemoveContainer" containerID="8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e" Jun 05 13:12:03.051950 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.051868 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:12:03.054727 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.054707 2570 scope.go:117] "RemoveContainer" containerID="35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927" Jun 05 13:12:03.062690 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.062669 2570 scope.go:117] "RemoveContainer" containerID="af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac" Jun 05 13:12:03.069848 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.069823 2570 scope.go:117] "RemoveContainer" containerID="812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1" Jun 05 13:12:03.070143 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.070117 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1\": container with ID starting with 812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1 not found: ID does not exist" containerID="812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1" Jun 05 13:12:03.070227 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070157 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1"} err="failed to get container status \"812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1\": rpc error: code = NotFound desc = could not find container \"812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1\": container with ID starting with 812153000e7f48ba37d7e34cc83cb996c85998cf06b90731c9b8f932899d23a1 not found: ID does not exist" Jun 05 13:12:03.070227 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070202 2570 scope.go:117] "RemoveContainer" containerID="249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e" Jun 05 13:12:03.070471 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.070450 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e\": container with ID starting with 249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e not found: ID does not exist" containerID="249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e" Jun 05 13:12:03.070511 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070478 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e"} err="failed to get container status \"249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e\": rpc error: code = NotFound desc = could not find container \"249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e\": container with ID starting with 249ea4e316076d4a224ec5e8a134c4ddc24b2c80403e9d0e78443f8ce2e56f3e not found: ID does not exist" Jun 05 13:12:03.070511 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070493 2570 scope.go:117] "RemoveContainer" containerID="91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e" Jun 05 13:12:03.070733 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.070716 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e\": container with ID starting with 91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e not found: ID does not exist" containerID="91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e" Jun 05 13:12:03.070776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070736 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e"} err="failed to get container status \"91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e\": rpc error: code = NotFound desc = could not find container \"91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e\": container with ID starting with 91ecf699503ad88d820bf56d2f52237d7de1e96b81ea0c7d66ed40163c2f424e not found: ID does not exist" Jun 05 13:12:03.070776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070757 2570 scope.go:117] "RemoveContainer" containerID="b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9" Jun 05 13:12:03.070971 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.070955 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9\": container with ID starting with b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9 not found: ID does not exist" containerID="b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9" Jun 05 13:12:03.071035 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.070981 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9"} err="failed to get container status \"b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9\": rpc error: code = NotFound desc = could not find container \"b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9\": container with ID starting with b1e821d1e7876d41928f7171bb7f78327be7aa28f29e850fe6dcd697de0003d9 not found: ID does not exist" Jun 05 13:12:03.071035 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.071001 2570 scope.go:117] "RemoveContainer" containerID="8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e" Jun 05 13:12:03.071211 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.071194 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e\": container with ID starting with 8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e not found: ID does not exist" containerID="8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e" Jun 05 13:12:03.071356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.071218 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e"} err="failed to get container status \"8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e\": rpc error: code = NotFound desc = could not find container \"8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e\": container with ID starting with 8fea162d9b80c942067e0a10e7551f71a430ded3429775780779b93d3751f17e not found: ID does not exist" Jun 05 13:12:03.071356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.071237 2570 scope.go:117] "RemoveContainer" containerID="35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927" Jun 05 13:12:03.071482 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.071466 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927\": container with ID starting with 35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927 not found: ID does not exist" containerID="35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927" Jun 05 13:12:03.071521 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.071488 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927"} err="failed to get container status \"35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927\": rpc error: code = NotFound desc = could not find container \"35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927\": container with ID starting with 35baee36ae137b8411e5c2a14e87306cad9a4ec718650c56e6801c36c15cb927 not found: ID does not exist" Jun 05 13:12:03.071521 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.071506 2570 scope.go:117] "RemoveContainer" containerID="af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac" Jun 05 13:12:03.071711 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:03.071696 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac\": container with ID starting with af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac not found: ID does not exist" containerID="af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac" Jun 05 13:12:03.071752 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.071715 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac"} err="failed to get container status \"af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac\": rpc error: code = NotFound desc = could not find container \"af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac\": container with ID starting with af95248343e5e5a96c7b82d91713cbec293f1521f7b9123d736c866731efa1ac not found: ID does not exist" Jun 05 13:12:03.079046 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.079024 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:12:03.152194 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152164 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:12:03.152493 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152481 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="init-config-reloader" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152495 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="init-config-reloader" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152503 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152508 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152517 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-thanos" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152522 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-thanos" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152530 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="thanos-sidecar" Jun 05 13:12:03.152533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152535 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="thanos-sidecar" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152544 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="config-reloader" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152551 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="config-reloader" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152560 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-web" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152565 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-web" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152574 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="prometheus" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152579 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="prometheus" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152626 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="prometheus" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152633 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="thanos-sidecar" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152638 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-web" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152647 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152653 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="config-reloader" Jun 05 13:12:03.152743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.152658 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" containerName="kube-rbac-proxy-thanos" Jun 05 13:12:03.158381 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.158359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.165533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.165515 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 05 13:12:03.169605 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.169581 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 05 13:12:03.169717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.169611 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 05 13:12:03.169717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.169671 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 05 13:12:03.169717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.169711 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 05 13:12:03.169881 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.169715 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 05 13:12:03.169881 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.169742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 05 13:12:03.170033 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.170017 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 05 13:12:03.170226 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.170210 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-chbjt\"" Jun 05 13:12:03.170298 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.170283 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 05 13:12:03.170418 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.170400 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 05 13:12:03.170510 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.170497 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cp24o9ojus79l\"" Jun 05 13:12:03.170565 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.170516 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 05 13:12:03.172201 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.172186 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 05 13:12:03.172866 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.172853 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 05 13:12:03.174333 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.174317 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:03.187666 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.187417 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188404 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188451 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188500 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188541 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188582 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-config\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188613 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188641 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188678 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188709 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188773 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188825 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8e7b14e-9514-44cc-acd1-05e4100cac67-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188860 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188884 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzb7r\" (UniqueName: \"kubernetes.io/projected/c8e7b14e-9514-44cc-acd1-05e4100cac67-kube-api-access-pzb7r\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188923 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.191703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188957 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8e7b14e-9514-44cc-acd1-05e4100cac67-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.192576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.188987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.192576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.189044 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.192576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.189082 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.192576 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.189111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.247556 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.247524 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:12:03.289718 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289676 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.289718 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289714 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.289934 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289736 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.289934 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289753 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.289934 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290102 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289944 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290102 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.289991 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-config\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290102 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290102 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290102 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290108 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290151 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290170 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290233 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8e7b14e-9514-44cc-acd1-05e4100cac67-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290307 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290333 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzb7r\" (UniqueName: \"kubernetes.io/projected/c8e7b14e-9514-44cc-acd1-05e4100cac67-kube-api-access-pzb7r\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290728 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290442 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290728 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290496 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290728 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8e7b14e-9514-44cc-acd1-05e4100cac67-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290728 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290728 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.290983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.290925 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.291171 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.291150 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.294350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.291737 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.294350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.294052 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8e7b14e-9514-44cc-acd1-05e4100cac67-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.294350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.294156 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.294350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.294209 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.294718 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.294436 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.294718 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.294598 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295310 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.294879 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295310 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295044 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e7b14e-9514-44cc-acd1-05e4100cac67-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295310 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295104 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295310 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295240 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8e7b14e-9514-44cc-acd1-05e4100cac67-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295310 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295293 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-config\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295616 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295344 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295616 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295390 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.295616 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.295506 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.296470 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.296451 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8e7b14e-9514-44cc-acd1-05e4100cac67-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.311187 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.311160 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzb7r\" (UniqueName: \"kubernetes.io/projected/c8e7b14e-9514-44cc-acd1-05e4100cac67-kube-api-access-pzb7r\") pod \"prometheus-k8s-0\" (UID: \"c8e7b14e-9514-44cc-acd1-05e4100cac67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.467220 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.467182 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:03.638888 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.638853 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 05 13:12:03.641911 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:12:03.641882 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e7b14e_9514_44cc_acd1_05e4100cac67.slice/crio-ff1484476828962a4c413623c20d5902bd34bef5011c1024848af2e78592e60d WatchSource:0}: Error finding container ff1484476828962a4c413623c20d5902bd34bef5011c1024848af2e78592e60d: Status 404 returned error can't find the container with id ff1484476828962a4c413623c20d5902bd34bef5011c1024848af2e78592e60d Jun 05 13:12:03.781841 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:03.781762 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a542702e-8139-4eac-bf3b-9315b6a60a05" path="/var/lib/kubelet/pods/a542702e-8139-4eac-bf3b-9315b6a60a05/volumes" Jun 05 13:12:04.022523 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:04.022490 2570 generic.go:356] "Generic (PLEG): container finished" podID="c8e7b14e-9514-44cc-acd1-05e4100cac67" containerID="3d1fc9db2f13ff3ab11fbe694a78449e64bca11122f8d3a8280f60eee8b18228" exitCode=0 Jun 05 13:12:04.022946 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:04.022578 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerDied","Data":"3d1fc9db2f13ff3ab11fbe694a78449e64bca11122f8d3a8280f60eee8b18228"} Jun 05 13:12:04.022946 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:04.022663 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"ff1484476828962a4c413623c20d5902bd34bef5011c1024848af2e78592e60d"} Jun 05 13:12:05.031817 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:05.031784 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"75f72d4a17279704416e33a53701d02b41a8f5414bf54b00071085e9cee6de9a"} Jun 05 13:12:05.032184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:05.031850 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"84a44a32f3cf0c849ae003bd2a54faa76681b215ef7b166aea6e001f5491aeff"} Jun 05 13:12:05.032184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:05.031879 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"54cefcdb2fb96f9c9ef2c988252b042513aab267eb51bf1b3c2a9112cd79c15d"} Jun 05 13:12:05.032184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:05.031903 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"097d6ce166c25aaece5bcbe4e300799e593778d7dfd2a22a3a67aa84f5453a13"} Jun 05 13:12:05.032184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:05.031928 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"1ab06416a6c5f15424afe928fd6a21f5d8ae5ea8a810e83f0fa2cd83272d2a70"} Jun 05 13:12:05.032184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:05.031952 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8e7b14e-9514-44cc-acd1-05e4100cac67","Type":"ContainerStarted","Data":"9b3c52c5dbe0d5377bfbed8e6ffd776e3a247eff0cee07bb4b290d9e9eb68d26"} Jun 05 13:12:08.467614 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:08.467570 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:12:08.760479 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:08.760442 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:08.860706 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:08.860672 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:14.669805 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:14.669769 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:14.804659 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:14.804625 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:20.059584 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.059490 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" containerName="registry" containerID="cri-o://8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1" gracePeriod=30 Jun 05 13:12:20.314577 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.314521 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:12:20.360932 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.360878 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=17.342341014 podStartE2EDuration="17.360861171s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.520157ms" isStatefulPod=false podCreationTimestamp="2026-06-05 13:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:12:05.094311915 +0000 UTC m=+337.735635716" watchObservedRunningTime="2026-06-05 13:12:20.360861171 +0000 UTC m=+353.002185012" Jun 05 13:12:20.445075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445039 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-trusted-ca\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445295 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445138 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7xll\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-kube-api-access-h7xll\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445295 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445192 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c5754e-c750-4fdf-a2e5-19a92fa6671e-ca-trust-extracted\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445295 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445220 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-bound-sa-token\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445295 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445280 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-image-registry-private-configuration\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445525 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445319 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445525 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445361 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-certificates\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445525 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445389 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-installation-pull-secrets\") pod \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\" (UID: \"64c5754e-c750-4fdf-a2e5-19a92fa6671e\") " Jun 05 13:12:20.445674 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445586 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:20.445725 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445706 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-trusted-ca\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.445899 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.445861 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:12:20.448041 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.448012 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:20.448155 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.448023 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-kube-api-access-h7xll" (OuterVolumeSpecName: "kube-api-access-h7xll") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "kube-api-access-h7xll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:12:20.448155 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.448043 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:12:20.448155 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.448111 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:12:20.448291 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.448266 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:12:20.453935 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.453912 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c5754e-c750-4fdf-a2e5-19a92fa6671e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "64c5754e-c750-4fdf-a2e5-19a92fa6671e" (UID: "64c5754e-c750-4fdf-a2e5-19a92fa6671e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 05 13:12:20.546452 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546412 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7xll\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-kube-api-access-h7xll\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.546452 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546445 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/64c5754e-c750-4fdf-a2e5-19a92fa6671e-ca-trust-extracted\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.546452 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546455 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-bound-sa-token\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.546691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546466 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-image-registry-private-configuration\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.546691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546478 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-tls\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.546691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546486 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/64c5754e-c750-4fdf-a2e5-19a92fa6671e-registry-certificates\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:20.546691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:20.546494 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/64c5754e-c750-4fdf-a2e5-19a92fa6671e-installation-pull-secrets\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:12:21.099002 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.098969 2570 generic.go:356] "Generic (PLEG): container finished" podID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" containerID="8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1" exitCode=0 Jun 05 13:12:21.099480 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.099013 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" Jun 05 13:12:21.099480 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.099022 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" event={"ID":"64c5754e-c750-4fdf-a2e5-19a92fa6671e","Type":"ContainerDied","Data":"8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1"} Jun 05 13:12:21.099480 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.099065 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f876c6468-qmzf4" event={"ID":"64c5754e-c750-4fdf-a2e5-19a92fa6671e","Type":"ContainerDied","Data":"bdb6bb1c6736b8648303572a5989dea635ac4440c032cbe5f80228441bec0403"} Jun 05 13:12:21.099480 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.099084 2570 scope.go:117] "RemoveContainer" containerID="8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1" Jun 05 13:12:21.108293 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.108271 2570 scope.go:117] "RemoveContainer" containerID="8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1" Jun 05 13:12:21.108571 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:12:21.108552 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1\": container with ID starting with 8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1 not found: ID does not exist" containerID="8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1" Jun 05 13:12:21.108619 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.108582 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1"} err="failed to get container status \"8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1\": rpc error: code = NotFound desc = could not find container \"8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1\": container with ID starting with 8ba1db8cd97ece918ff0fb7aa98e72c7682cc5ddf1b6c3afd457fd753f78cbc1 not found: ID does not exist" Jun 05 13:12:21.124921 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.124890 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f876c6468-qmzf4"] Jun 05 13:12:21.133825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.133794 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f876c6468-qmzf4"] Jun 05 13:12:21.782187 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:21.782140 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" path="/var/lib/kubelet/pods/64c5754e-c750-4fdf-a2e5-19a92fa6671e/volumes" Jun 05 13:12:53.612528 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:53.612496 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:12:53.680230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:12:53.680198 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:03.322707 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:03.322665 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:03.467955 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:03.467923 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:13:03.483413 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:03.483386 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:13:04.284816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:04.284778 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 05 13:13:15.570703 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:15.570645 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:15.657177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:15.657143 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:31.516210 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:31.516170 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:50.299230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:50.299143 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:51.293944 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:51.293905 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:54.490966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:54.490918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:54.592416 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:54.592381 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:55.799821 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:55.799778 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:56.026475 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:56.026440 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:56.244308 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:56.244268 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:59.511536 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:59.511501 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:13:59.750702 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:13:59.750663 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:01.444056 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.444009 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn"] Jun 05 13:14:01.444518 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.444343 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" containerName="registry" Jun 05 13:14:01.444518 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.444355 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" containerName="registry" Jun 05 13:14:01.444518 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.444417 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c5754e-c750-4fdf-a2e5-19a92fa6671e" containerName="registry" Jun 05 13:14:01.447335 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.447319 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.456692 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.456670 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 05 13:14:01.457632 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.457618 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-szmz7\"" Jun 05 13:14:01.457682 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.457631 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 05 13:14:01.474061 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.474031 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:01.482607 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.482575 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn"] Jun 05 13:14:01.637692 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.637655 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:01.640826 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.640796 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b96fda52-44bc-4143-a5b8-0364f3696a09-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-x4wkn\" (UID: \"b96fda52-44bc-4143-a5b8-0364f3696a09\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.640907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.640840 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2rdx\" (UniqueName: \"kubernetes.io/projected/b96fda52-44bc-4143-a5b8-0364f3696a09-kube-api-access-v2rdx\") pod \"cert-manager-cainjector-5d99449ffd-x4wkn\" (UID: \"b96fda52-44bc-4143-a5b8-0364f3696a09\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.742218 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.742179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b96fda52-44bc-4143-a5b8-0364f3696a09-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-x4wkn\" (UID: \"b96fda52-44bc-4143-a5b8-0364f3696a09\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.742406 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.742222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2rdx\" (UniqueName: \"kubernetes.io/projected/b96fda52-44bc-4143-a5b8-0364f3696a09-kube-api-access-v2rdx\") pod \"cert-manager-cainjector-5d99449ffd-x4wkn\" (UID: \"b96fda52-44bc-4143-a5b8-0364f3696a09\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.762802 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.762769 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2rdx\" (UniqueName: \"kubernetes.io/projected/b96fda52-44bc-4143-a5b8-0364f3696a09-kube-api-access-v2rdx\") pod \"cert-manager-cainjector-5d99449ffd-x4wkn\" (UID: \"b96fda52-44bc-4143-a5b8-0364f3696a09\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.766949 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.766922 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b96fda52-44bc-4143-a5b8-0364f3696a09-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-x4wkn\" (UID: \"b96fda52-44bc-4143-a5b8-0364f3696a09\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.767835 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.767814 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" Jun 05 13:14:01.929516 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.929480 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn"] Jun 05 13:14:01.932773 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:14:01.932744 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb96fda52_44bc_4143_a5b8_0364f3696a09.slice/crio-425413bb1154916b467e0658c352773c3b03bb4e890160d4b4ef0ab3107c473f WatchSource:0}: Error finding container 425413bb1154916b467e0658c352773c3b03bb4e890160d4b4ef0ab3107c473f: Status 404 returned error can't find the container with id 425413bb1154916b467e0658c352773c3b03bb4e890160d4b4ef0ab3107c473f Jun 05 13:14:01.934769 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:01.934755 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 05 13:14:02.501964 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:02.501924 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" event={"ID":"b96fda52-44bc-4143-a5b8-0364f3696a09","Type":"ContainerStarted","Data":"425413bb1154916b467e0658c352773c3b03bb4e890160d4b4ef0ab3107c473f"} Jun 05 13:14:05.455679 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:05.455647 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:05.516711 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:05.516674 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" event={"ID":"b96fda52-44bc-4143-a5b8-0364f3696a09","Type":"ContainerStarted","Data":"41ac6c289dfc20e115581a5e2adc78d405d394ce3e591081e9f0242a8c25f578"} Jun 05 13:14:05.578398 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:05.578353 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:07.856713 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:07.856676 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:08.265486 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:08.265444 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:08.418181 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:08.418131 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:14.157388 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:14.157349 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:14.264038 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:14.263988 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:16.080872 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:16.080830 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:27.593663 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:27.593631 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:27.693893 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:27.693864 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:28.204397 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.204365 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:28.557717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.557687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:28.660350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.660315 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:28.723559 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.723504 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-x4wkn" podStartSLOduration=24.921618198 podStartE2EDuration="27.723487066s" totalImagesPullingTime="2.801868868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:14:05.553934048 +0000 UTC m=+458.195257849" watchObservedRunningTime="2026-06-05 13:14:28.723487066 +0000 UTC m=+481.364810869" Jun 05 13:14:28.723784 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.723771 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv"] Jun 05 13:14:28.727189 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.727170 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.732785 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.732702 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 05 13:14:28.733232 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.733198 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 05 13:14:28.733232 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.733212 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 05 13:14:28.733434 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.733234 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-49q62\"" Jun 05 13:14:28.733434 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.733212 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 05 13:14:28.745306 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.745283 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv"] Jun 05 13:14:28.766857 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.766828 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/855c3beb-cda0-4b33-a930-c53c4c2b4310-tmp\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.766992 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.766876 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/855c3beb-cda0-4b33-a930-c53c4c2b4310-manifests\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.766992 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.766906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9k97\" (UniqueName: \"kubernetes.io/projected/855c3beb-cda0-4b33-a930-c53c4c2b4310-kube-api-access-m9k97\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.766992 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.766936 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/855c3beb-cda0-4b33-a930-c53c4c2b4310-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.766992 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.766955 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/855c3beb-cda0-4b33-a930-c53c4c2b4310-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.867816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.867729 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/855c3beb-cda0-4b33-a930-c53c4c2b4310-manifests\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.867816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.867769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9k97\" (UniqueName: \"kubernetes.io/projected/855c3beb-cda0-4b33-a930-c53c4c2b4310-kube-api-access-m9k97\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.867816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.867792 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/855c3beb-cda0-4b33-a930-c53c4c2b4310-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.868035 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.867905 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/855c3beb-cda0-4b33-a930-c53c4c2b4310-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.868035 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.868016 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/855c3beb-cda0-4b33-a930-c53c4c2b4310-tmp\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.868175 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.868156 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/855c3beb-cda0-4b33-a930-c53c4c2b4310-manifests\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.870296 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.870267 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/855c3beb-cda0-4b33-a930-c53c4c2b4310-tmp\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.870463 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.870443 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/855c3beb-cda0-4b33-a930-c53c4c2b4310-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.870525 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.870474 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/855c3beb-cda0-4b33-a930-c53c4c2b4310-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.882791 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.882767 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9k97\" (UniqueName: \"kubernetes.io/projected/855c3beb-cda0-4b33-a930-c53c4c2b4310-kube-api-access-m9k97\") pod \"opendatahub-operator-controller-manager-58b9f446d7-t69wv\" (UID: \"855c3beb-cda0-4b33-a930-c53c4c2b4310\") " pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:28.931969 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:28.931931 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:29.033959 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.033910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:29.037949 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.037928 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:29.130614 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.130576 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:29.176278 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.176229 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv"] Jun 05 13:14:29.179807 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:14:29.179782 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855c3beb_cda0_4b33_a930_c53c4c2b4310.slice/crio-a2a8f47d771b8b7e5c018c79a2a36ff838bca9a4d7a2b990bbd12b6014399e1f WatchSource:0}: Error finding container a2a8f47d771b8b7e5c018c79a2a36ff838bca9a4d7a2b990bbd12b6014399e1f: Status 404 returned error can't find the container with id a2a8f47d771b8b7e5c018c79a2a36ff838bca9a4d7a2b990bbd12b6014399e1f Jun 05 13:14:29.230034 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.230003 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:29.331568 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.331539 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:29.432889 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.432771 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:29.623286 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:29.623234 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" event={"ID":"855c3beb-cda0-4b33-a930-c53c4c2b4310","Type":"ContainerStarted","Data":"a2a8f47d771b8b7e5c018c79a2a36ff838bca9a4d7a2b990bbd12b6014399e1f"} Jun 05 13:14:31.592739 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:31.592691 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:31.697715 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:31.697675 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:32.640493 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:32.640460 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" event={"ID":"855c3beb-cda0-4b33-a930-c53c4c2b4310","Type":"ContainerStarted","Data":"d0618b05357410f598976056f14db96acba80cee4d34e76465dc2b3ae5fed2f3"} Jun 05 13:14:33.646906 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:33.646869 2570 generic.go:356] "Generic (PLEG): container finished" podID="855c3beb-cda0-4b33-a930-c53c4c2b4310" containerID="d0618b05357410f598976056f14db96acba80cee4d34e76465dc2b3ae5fed2f3" exitCode=0 Jun 05 13:14:33.647384 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:33.646950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" event={"ID":"855c3beb-cda0-4b33-a930-c53c4c2b4310","Type":"ContainerDied","Data":"d0618b05357410f598976056f14db96acba80cee4d34e76465dc2b3ae5fed2f3"} Jun 05 13:14:34.653320 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:34.653283 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" event={"ID":"855c3beb-cda0-4b33-a930-c53c4c2b4310","Type":"ContainerStarted","Data":"a5058c7ead5e6687bb9901979cf9271001982a72419505f4c2b800fde012a173"} Jun 05 13:14:34.653708 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:34.653431 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:34.687820 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:34.687772 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" podStartSLOduration=2.512646861 podStartE2EDuration="6.687756387s" totalImagesPullingTime="3.206733743s" totalInitContainerRuntime="968.375783ms" isStatefulPod=false podCreationTimestamp="2026-06-05 13:14:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:14:34.686608774 +0000 UTC m=+487.327932609" watchObservedRunningTime="2026-06-05 13:14:34.687756387 +0000 UTC m=+487.329080189" Jun 05 13:14:37.333207 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:37.333168 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:37.442261 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:37.442223 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:42.378653 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:42.378624 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:44.684465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:44.684429 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:44.771935 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:44.771892 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:45.087689 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:45.087646 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:45.659972 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:45.659944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58b9f446d7-t69wv" Jun 05 13:14:47.503812 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:47.503770 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:47.658462 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:47.658427 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:48.570420 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:48.570389 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:48.712489 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:48.712457 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:48.882438 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:48.882351 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:49.349445 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:49.349408 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:49.450532 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:49.450493 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:54.096155 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:54.096117 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:54.187096 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:54.187063 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:55.872166 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.872122 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql"] Jun 05 13:14:55.876368 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.876341 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:55.881393 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.881187 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 05 13:14:55.883145 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.882742 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 05 13:14:55.883145 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.882822 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 05 13:14:55.883145 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.882826 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 05 13:14:55.883145 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.882970 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h9r5c\"" Jun 05 13:14:55.883145 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.883115 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 05 13:14:55.892590 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.892563 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql"] Jun 05 13:14:55.912587 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.912554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d41eada-d384-49fb-b319-d8fbc1b8726a-metrics-cert\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:55.912775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.912593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d41eada-d384-49fb-b319-d8fbc1b8726a-cert\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:55.912775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.912666 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6d41eada-d384-49fb-b319-d8fbc1b8726a-manager-config\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:55.912775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:55.912696 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djzsx\" (UniqueName: \"kubernetes.io/projected/6d41eada-d384-49fb-b319-d8fbc1b8726a-kube-api-access-djzsx\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.013892 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.013853 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6d41eada-d384-49fb-b319-d8fbc1b8726a-manager-config\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.013892 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.013900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djzsx\" (UniqueName: \"kubernetes.io/projected/6d41eada-d384-49fb-b319-d8fbc1b8726a-kube-api-access-djzsx\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.014158 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.014027 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d41eada-d384-49fb-b319-d8fbc1b8726a-metrics-cert\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.014158 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.014078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d41eada-d384-49fb-b319-d8fbc1b8726a-cert\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.014650 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.014627 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6d41eada-d384-49fb-b319-d8fbc1b8726a-manager-config\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.016465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.016445 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d41eada-d384-49fb-b319-d8fbc1b8726a-metrics-cert\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.016545 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.016506 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d41eada-d384-49fb-b319-d8fbc1b8726a-cert\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.026007 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.025980 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djzsx\" (UniqueName: \"kubernetes.io/projected/6d41eada-d384-49fb-b319-d8fbc1b8726a-kube-api-access-djzsx\") pod \"lws-controller-manager-b5dbd5dd-plrql\" (UID: \"6d41eada-d384-49fb-b319-d8fbc1b8726a\") " pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.190645 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.190553 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:56.327640 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.327611 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql"] Jun 05 13:14:56.331738 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:14:56.331713 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d41eada_d384_49fb_b319_d8fbc1b8726a.slice/crio-f06b7a38143a9057ce3bc04d4c0905a768c91374b53700c524e245358bfff1db WatchSource:0}: Error finding container f06b7a38143a9057ce3bc04d4c0905a768c91374b53700c524e245358bfff1db: Status 404 returned error can't find the container with id f06b7a38143a9057ce3bc04d4c0905a768c91374b53700c524e245358bfff1db Jun 05 13:14:56.751705 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:56.751668 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" event={"ID":"6d41eada-d384-49fb-b319-d8fbc1b8726a","Type":"ContainerStarted","Data":"f06b7a38143a9057ce3bc04d4c0905a768c91374b53700c524e245358bfff1db"} Jun 05 13:14:58.432528 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.432471 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:58.959167 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.959132 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v"] Jun 05 13:14:58.963205 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.963154 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:58.969031 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.969005 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 05 13:14:58.969293 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.969276 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 05 13:14:58.969556 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.969449 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 05 13:14:58.970101 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.969962 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 05 13:14:58.970101 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.970010 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-smmhk\"" Jun 05 13:14:58.983168 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:58.983139 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v"] Jun 05 13:14:59.039132 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.039094 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9054225c-7b02-48a0-983e-fb671bbc58bd-tls-certs\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.039308 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.039201 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4sdj\" (UniqueName: \"kubernetes.io/projected/9054225c-7b02-48a0-983e-fb671bbc58bd-kube-api-access-b4sdj\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.039308 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.039273 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9054225c-7b02-48a0-983e-fb671bbc58bd-tmp\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.140556 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.140530 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sdj\" (UniqueName: \"kubernetes.io/projected/9054225c-7b02-48a0-983e-fb671bbc58bd-kube-api-access-b4sdj\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.140681 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.140580 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9054225c-7b02-48a0-983e-fb671bbc58bd-tmp\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.140681 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.140670 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9054225c-7b02-48a0-983e-fb671bbc58bd-tls-certs\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.142814 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.142791 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9054225c-7b02-48a0-983e-fb671bbc58bd-tmp\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.143028 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.143007 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9054225c-7b02-48a0-983e-fb671bbc58bd-tls-certs\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.163082 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.163060 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4sdj\" (UniqueName: \"kubernetes.io/projected/9054225c-7b02-48a0-983e-fb671bbc58bd-kube-api-access-b4sdj\") pod \"kube-auth-proxy-f65db7bc7-8jh2v\" (UID: \"9054225c-7b02-48a0-983e-fb671bbc58bd\") " pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.277503 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.277465 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" Jun 05 13:14:59.419312 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.419275 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v"] Jun 05 13:14:59.426274 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:14:59.426224 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9054225c_7b02_48a0_983e_fb671bbc58bd.slice/crio-24bcbb83487dd0f6b58116745c9b73a0d85cb49d4f1e983a271f4c54b2bca1eb WatchSource:0}: Error finding container 24bcbb83487dd0f6b58116745c9b73a0d85cb49d4f1e983a271f4c54b2bca1eb: Status 404 returned error can't find the container with id 24bcbb83487dd0f6b58116745c9b73a0d85cb49d4f1e983a271f4c54b2bca1eb Jun 05 13:14:59.638484 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.638403 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:14:59.767302 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.767267 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" event={"ID":"9054225c-7b02-48a0-983e-fb671bbc58bd","Type":"ContainerStarted","Data":"24bcbb83487dd0f6b58116745c9b73a0d85cb49d4f1e983a271f4c54b2bca1eb"} Jun 05 13:14:59.768690 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.768669 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" event={"ID":"6d41eada-d384-49fb-b319-d8fbc1b8726a","Type":"ContainerStarted","Data":"495ca59b5d4449e62ad146ea8340ef21ccefa5410e669794075988843ae4a9cd"} Jun 05 13:14:59.768819 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.768804 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:14:59.795158 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.795108 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" podStartSLOduration=1.9998570230000001 podStartE2EDuration="4.795090766s" totalImagesPullingTime="2.795233743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:14:59.793653185 +0000 UTC m=+512.434976986" watchObservedRunningTime="2026-06-05 13:14:59.795090766 +0000 UTC m=+512.436414567" Jun 05 13:14:59.957193 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:14:59.957100 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:00.084385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:00.084344 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:00.172643 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:00.172609 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:02.785237 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:02.785200 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" event={"ID":"9054225c-7b02-48a0-983e-fb671bbc58bd","Type":"ContainerStarted","Data":"de81b7c6923d27249671b0ba10899d7afa1f312afd72cdbcd574a41d015947e2"} Jun 05 13:15:02.823491 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:02.823436 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f65db7bc7-8jh2v" podStartSLOduration=2.121793831 podStartE2EDuration="4.823419282s" totalImagesPullingTime="2.701625451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:15:02.821783472 +0000 UTC m=+515.463107274" watchObservedRunningTime="2026-06-05 13:15:02.823419282 +0000 UTC m=+515.464743083" Jun 05 13:15:03.830474 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:03.830443 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:03.908107 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:03.908075 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:08.792260 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:08.792221 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:08.868397 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:08.868361 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:10.776470 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:10.776442 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b5dbd5dd-plrql" Jun 05 13:15:11.968657 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:11.968623 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:12.067919 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:12.067882 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:13.805202 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:13.805165 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:13.913591 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:13.913538 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:14.521145 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:14.521109 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:14.635203 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:14.635173 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:15.931557 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:15.931521 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:30.756138 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:30.756062 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:30.862401 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:30.862371 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:31.904469 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:31.904431 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:34.369049 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:34.369018 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:34.586085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:34.586049 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:34.891269 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:34.891218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:35.087493 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:35.087457 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:35.186691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:35.186614 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:38.125523 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:38.125486 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:49.119875 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:49.119841 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:49.217867 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:49.217838 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:49.327731 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:49.327697 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:58.017519 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:58.017486 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:58.112141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:58.112106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:58.178230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:58.178197 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:58.735603 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:58.735560 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:15:58.822348 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:15:58.822313 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:09.088928 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:09.088896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:09.158753 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:09.158719 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:09.220699 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:09.220668 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:17.545211 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:17.545177 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:17.951504 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:17.951420 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:18.556273 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:18.556221 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:18.712881 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:18.712835 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:22.899231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:22.899193 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:25.227323 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:25.227280 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:26.657607 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.657573 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:26.667677 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.667651 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5"] Jun 05 13:16:26.671157 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.671140 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.677892 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.677778 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 05 13:16:26.677967 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.677891 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 05 13:16:26.678040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.678018 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wkh84\"" Jun 05 13:16:26.740818 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.740772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5"] Jun 05 13:16:26.741232 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.741202 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98cwd\" (UniqueName: \"kubernetes.io/projected/a6f5a778-8cfa-48f0-94af-2c3a86929a56-kube-api-access-98cwd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.741420 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.741398 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f5a778-8cfa-48f0-94af-2c3a86929a56-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.841896 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.841863 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f5a778-8cfa-48f0-94af-2c3a86929a56-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.842073 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.842005 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98cwd\" (UniqueName: \"kubernetes.io/projected/a6f5a778-8cfa-48f0-94af-2c3a86929a56-kube-api-access-98cwd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.842236 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.842220 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f5a778-8cfa-48f0-94af-2c3a86929a56-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.862511 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.862485 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98cwd\" (UniqueName: \"kubernetes.io/projected/a6f5a778-8cfa-48f0-94af-2c3a86929a56-kube-api-access-98cwd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:26.907363 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.907332 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:26.981179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:26.981153 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:27.162302 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:27.162272 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5"] Jun 05 13:16:27.165590 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:16:27.165562 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f5a778_8cfa_48f0_94af_2c3a86929a56.slice/crio-5487c6a064d824b1a120986a3f08da37ba7b1397826000f52f988d85db9d8ac1 WatchSource:0}: Error finding container 5487c6a064d824b1a120986a3f08da37ba7b1397826000f52f988d85db9d8ac1: Status 404 returned error can't find the container with id 5487c6a064d824b1a120986a3f08da37ba7b1397826000f52f988d85db9d8ac1 Jun 05 13:16:27.764919 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:27.764891 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:16:27.766654 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:27.766632 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:16:27.824938 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:27.824902 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:28.171197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:28.171120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" event={"ID":"a6f5a778-8cfa-48f0-94af-2c3a86929a56","Type":"ContainerStarted","Data":"5487c6a064d824b1a120986a3f08da37ba7b1397826000f52f988d85db9d8ac1"} Jun 05 13:16:29.044098 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:29.044045 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:30.445946 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:30.445908 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:32.256724 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:32.256672 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:32.460390 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:32.460340 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:32.606107 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:32.606024 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:34.204566 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:34.204528 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" event={"ID":"a6f5a778-8cfa-48f0-94af-2c3a86929a56","Type":"ContainerStarted","Data":"228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734"} Jun 05 13:16:34.204985 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:34.204684 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:34.250092 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:34.250037 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" podStartSLOduration=2.256742399 podStartE2EDuration="8.250020508s" totalImagesPullingTime="5.993278109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:16:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:16:34.248822189 +0000 UTC m=+606.890146001" watchObservedRunningTime="2026-06-05 13:16:34.250020508 +0000 UTC m=+606.891344308" Jun 05 13:16:36.338783 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:36.338735 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:36.509599 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:36.509568 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:37.344751 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:37.344708 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:37.661407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:37.661308 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:43.833896 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:43.833858 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:43.977868 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:43.977827 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:44.232952 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:44.232922 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:44.356350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:44.356320 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:45.212158 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:45.212130 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:47.221507 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.221470 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5"] Jun 05 13:16:47.221894 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.221697 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" containerName="manager" containerID="cri-o://228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734" gracePeriod=2 Jun 05 13:16:47.231127 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.231101 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5"] Jun 05 13:16:47.302446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.302418 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s"] Jun 05 13:16:47.302782 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.302770 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" containerName="manager" Jun 05 13:16:47.302832 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.302783 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" containerName="manager" Jun 05 13:16:47.302872 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.302863 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" containerName="manager" Jun 05 13:16:47.305893 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.305875 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.317780 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:16:47.317749 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:47.363791 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.363754 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s"] Jun 05 13:16:47.437602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.437525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n2c7s\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.437602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.437583 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6mgs\" (UniqueName: \"kubernetes.io/projected/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-kube-api-access-k6mgs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n2c7s\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.465897 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.465861 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:47.469832 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:16:47.469806 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:47.538326 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.538292 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98cwd\" (UniqueName: \"kubernetes.io/projected/a6f5a778-8cfa-48f0-94af-2c3a86929a56-kube-api-access-98cwd\") pod \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " Jun 05 13:16:47.538326 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.538330 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f5a778-8cfa-48f0-94af-2c3a86929a56-extensions-socket-volume\") pod \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\" (UID: \"a6f5a778-8cfa-48f0-94af-2c3a86929a56\") " Jun 05 13:16:47.538542 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.538434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n2c7s\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.538542 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.538474 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mgs\" (UniqueName: \"kubernetes.io/projected/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-kube-api-access-k6mgs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n2c7s\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.538840 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.538817 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n2c7s\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.538898 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.538871 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f5a778-8cfa-48f0-94af-2c3a86929a56-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a6f5a778-8cfa-48f0-94af-2c3a86929a56" (UID: "a6f5a778-8cfa-48f0-94af-2c3a86929a56"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 05 13:16:47.540642 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.540621 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6f5a778-8cfa-48f0-94af-2c3a86929a56-kube-api-access-98cwd" (OuterVolumeSpecName: "kube-api-access-98cwd") pod "a6f5a778-8cfa-48f0-94af-2c3a86929a56" (UID: "a6f5a778-8cfa-48f0-94af-2c3a86929a56"). InnerVolumeSpecName "kube-api-access-98cwd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:16:47.552923 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.552898 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6mgs\" (UniqueName: \"kubernetes.io/projected/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-kube-api-access-k6mgs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n2c7s\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.616591 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.616548 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:47.638981 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.638944 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98cwd\" (UniqueName: \"kubernetes.io/projected/a6f5a778-8cfa-48f0-94af-2c3a86929a56-kube-api-access-98cwd\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:16:47.638981 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.638974 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f5a778-8cfa-48f0-94af-2c3a86929a56-extensions-socket-volume\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:16:47.780591 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:16:47.780548 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0aa895_aa6a_49f3_bb32_1f5b4bd92b39.slice/crio-3baee47bbce771f802cc08c148211166b62f4decea3f86257d17ae988bdd4857 WatchSource:0}: Error finding container 3baee47bbce771f802cc08c148211166b62f4decea3f86257d17ae988bdd4857: Status 404 returned error can't find the container with id 3baee47bbce771f802cc08c148211166b62f4decea3f86257d17ae988bdd4857 Jun 05 13:16:47.781858 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:16:47.781824 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gkbx5\" is forbidden: User \"system:node:ip-10-0-141-104.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-104.ec2.internal' and this object" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:47.782735 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.782712 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" path="/var/lib/kubelet/pods/a6f5a778-8cfa-48f0-94af-2c3a86929a56/volumes" Jun 05 13:16:47.783074 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:47.783061 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s"] Jun 05 13:16:48.269643 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.269608 2570 generic.go:356] "Generic (PLEG): container finished" podID="a6f5a778-8cfa-48f0-94af-2c3a86929a56" containerID="228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734" exitCode=0 Jun 05 13:16:48.270060 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.269656 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gkbx5" Jun 05 13:16:48.270060 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.269705 2570 scope.go:117] "RemoveContainer" containerID="228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734" Jun 05 13:16:48.271616 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.271590 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" event={"ID":"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39","Type":"ContainerStarted","Data":"659fd58e7d792288c832aa49c1b13055ee195ef13d2ab93f79b46c022d834b36"} Jun 05 13:16:48.271744 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.271644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" event={"ID":"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39","Type":"ContainerStarted","Data":"3baee47bbce771f802cc08c148211166b62f4decea3f86257d17ae988bdd4857"} Jun 05 13:16:48.271899 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.271878 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:16:48.278386 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.278370 2570 scope.go:117] "RemoveContainer" containerID="228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734" Jun 05 13:16:48.278628 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:16:48.278609 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734\": container with ID starting with 228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734 not found: ID does not exist" containerID="228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734" Jun 05 13:16:48.278690 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.278640 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734"} err="failed to get container status \"228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734\": rpc error: code = NotFound desc = could not find container \"228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734\": container with ID starting with 228ccde5e6bf744886bee5db397c2e1811dc0b8cec7b490dedb5547b75584734 not found: ID does not exist" Jun 05 13:16:48.299303 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.299235 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" podStartSLOduration=1.2992199389999999 podStartE2EDuration="1.299219939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:16:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:16:48.296538013 +0000 UTC m=+620.937861825" watchObservedRunningTime="2026-06-05 13:16:48.299219939 +0000 UTC m=+620.940543740" Jun 05 13:16:48.819242 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.819210 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:48.932853 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:48.932816 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:52.786809 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:52.786770 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:52.926076 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:52.926040 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:53.459659 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:53.459625 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:53.590517 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:53.590485 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:56.999291 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:56.999237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:57.100444 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:57.100407 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:16:59.279740 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:16:59.279700 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:17:03.150328 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:03.150275 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:03.280013 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:03.279975 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:07.874070 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:07.874037 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:08.083739 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:08.083701 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:16.278202 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.278164 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s"] Jun 05 13:17:16.278614 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.278440 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" podUID="5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" containerName="manager" containerID="cri-o://659fd58e7d792288c832aa49c1b13055ee195ef13d2ab93f79b46c022d834b36" gracePeriod=10 Jun 05 13:17:16.402397 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.402367 2570 generic.go:356] "Generic (PLEG): container finished" podID="5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" containerID="659fd58e7d792288c832aa49c1b13055ee195ef13d2ab93f79b46c022d834b36" exitCode=0 Jun 05 13:17:16.402544 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.402448 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" event={"ID":"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39","Type":"ContainerDied","Data":"659fd58e7d792288c832aa49c1b13055ee195ef13d2ab93f79b46c022d834b36"} Jun 05 13:17:16.532094 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.532037 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:17:16.592519 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.592487 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-extensions-socket-volume\") pod \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " Jun 05 13:17:16.592678 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.592580 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6mgs\" (UniqueName: \"kubernetes.io/projected/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-kube-api-access-k6mgs\") pod \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\" (UID: \"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39\") " Jun 05 13:17:16.592874 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.592851 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" (UID: "5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 05 13:17:16.594708 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.594685 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-kube-api-access-k6mgs" (OuterVolumeSpecName: "kube-api-access-k6mgs") pod "5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" (UID: "5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39"). InnerVolumeSpecName "kube-api-access-k6mgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:17:16.694142 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.694108 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6mgs\" (UniqueName: \"kubernetes.io/projected/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-kube-api-access-k6mgs\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:17:16.694142 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:16.694133 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39-extensions-socket-volume\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:17:17.408424 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:17.408397 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" Jun 05 13:17:17.408825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:17.408437 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s" event={"ID":"5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39","Type":"ContainerDied","Data":"3baee47bbce771f802cc08c148211166b62f4decea3f86257d17ae988bdd4857"} Jun 05 13:17:17.408825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:17.408493 2570 scope.go:117] "RemoveContainer" containerID="659fd58e7d792288c832aa49c1b13055ee195ef13d2ab93f79b46c022d834b36" Jun 05 13:17:17.458631 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:17.458599 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s"] Jun 05 13:17:17.476159 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:17.476134 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n2c7s"] Jun 05 13:17:17.782380 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:17.782353 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" path="/var/lib/kubelet/pods/5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39/volumes" Jun 05 13:17:32.736266 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.736217 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd"] Jun 05 13:17:32.736620 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.736560 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" containerName="manager" Jun 05 13:17:32.736620 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.736571 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" containerName="manager" Jun 05 13:17:32.736691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.736634 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a0aa895-aa6a-49f3-bb32-1f5b4bd92b39" containerName="manager" Jun 05 13:17:32.741016 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.740998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.745945 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.745918 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 05 13:17:32.746069 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.746034 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7qkrw\"" Jun 05 13:17:32.815123 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.815084 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:32.820582 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.820557 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd"] Jun 05 13:17:32.839280 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839232 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt28m\" (UniqueName: \"kubernetes.io/projected/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-kube-api-access-jt28m\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839326 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839400 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839355 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839533 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839495 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839595 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839568 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.839635 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.839621 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.914594 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.914558 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:32.940608 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940577 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940760 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940760 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940760 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940760 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940784 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt28m\" (UniqueName: \"kubernetes.io/projected/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-kube-api-access-jt28m\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.940966 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.940868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.941178 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.941153 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.941239 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.941176 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.941316 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.941294 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.941483 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.941462 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.941623 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.941548 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.942988 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.942967 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.943209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.943192 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.952003 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.951975 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:32.952421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:32.952400 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt28m\" (UniqueName: \"kubernetes.io/projected/8163f9c8-dcf3-4d8e-807d-0b40b2b87048-kube-api-access-jt28m\") pod \"maas-default-gateway-openshift-default-687ff6996-zxjrd\" (UID: \"8163f9c8-dcf3-4d8e-807d-0b40b2b87048\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:33.053597 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:33.053511 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:33.195228 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:33.195200 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd"] Jun 05 13:17:33.197807 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:17:33.197778 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8163f9c8_dcf3_4d8e_807d_0b40b2b87048.slice/crio-7c9f5d27c6a563428506a3625d20d5c37371dd2777bc5c596df812826903cd58 WatchSource:0}: Error finding container 7c9f5d27c6a563428506a3625d20d5c37371dd2777bc5c596df812826903cd58: Status 404 returned error can't find the container with id 7c9f5d27c6a563428506a3625d20d5c37371dd2777bc5c596df812826903cd58 Jun 05 13:17:33.482360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:33.482322 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" event={"ID":"8163f9c8-dcf3-4d8e-807d-0b40b2b87048","Type":"ContainerStarted","Data":"7c9f5d27c6a563428506a3625d20d5c37371dd2777bc5c596df812826903cd58"} Jun 05 13:17:36.705040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:36.705010 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:36.878046 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:36.877999 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 05 13:17:36.878156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:36.878098 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 05 13:17:36.878156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:36.878145 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 05 13:17:37.507688 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:37.507652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" event={"ID":"8163f9c8-dcf3-4d8e-807d-0b40b2b87048","Type":"ContainerStarted","Data":"21bda51947740c81dfb974282360bc14077a4b73ddc4e1cfbb5d1b92f89f4274"} Jun 05 13:17:37.529498 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:37.529443 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" podStartSLOduration=1.8514363729999999 podStartE2EDuration="5.529427336s" totalImagesPullingTime="3.677990963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:17:37.528160963 +0000 UTC m=+670.169484844" watchObservedRunningTime="2026-06-05 13:17:37.529427336 +0000 UTC m=+670.170751137" Jun 05 13:17:38.034853 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.034818 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:38.035236 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.034983 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:38.038556 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.038539 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.041419 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.041386 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7rswj\"" Jun 05 13:17:38.041590 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.041572 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 05 13:17:38.042464 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.042441 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 05 13:17:38.042464 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.042460 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 05 13:17:38.054081 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.054053 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:38.054847 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.054826 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:38.059485 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.059461 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:38.123201 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.123168 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:38.127801 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.127773 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:38.188906 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.188863 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hknv\" (UniqueName: \"kubernetes.io/projected/4273eacb-eadf-4bae-ba8d-d0f98f6096af-kube-api-access-5hknv\") pod \"limitador-limitador-64c94fd64f-gf6sb\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.189141 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.188990 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4273eacb-eadf-4bae-ba8d-d0f98f6096af-config-file\") pod \"limitador-limitador-64c94fd64f-gf6sb\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.207347 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.207316 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:38.290069 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.289965 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hknv\" (UniqueName: \"kubernetes.io/projected/4273eacb-eadf-4bae-ba8d-d0f98f6096af-kube-api-access-5hknv\") pod \"limitador-limitador-64c94fd64f-gf6sb\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.290238 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.290083 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4273eacb-eadf-4bae-ba8d-d0f98f6096af-config-file\") pod \"limitador-limitador-64c94fd64f-gf6sb\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.290743 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.290723 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4273eacb-eadf-4bae-ba8d-d0f98f6096af-config-file\") pod \"limitador-limitador-64c94fd64f-gf6sb\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.300059 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.300032 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hknv\" (UniqueName: \"kubernetes.io/projected/4273eacb-eadf-4bae-ba8d-d0f98f6096af-kube-api-access-5hknv\") pod \"limitador-limitador-64c94fd64f-gf6sb\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.348919 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.348859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:38.488006 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.487981 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:38.489780 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:17:38.489754 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4273eacb_eadf_4bae_ba8d_d0f98f6096af.slice/crio-23e75e0b0ac38078bff83dca106d18dd4ca6c42a4a0f5024fa3df9a53d2169ff WatchSource:0}: Error finding container 23e75e0b0ac38078bff83dca106d18dd4ca6c42a4a0f5024fa3df9a53d2169ff: Status 404 returned error can't find the container with id 23e75e0b0ac38078bff83dca106d18dd4ca6c42a4a0f5024fa3df9a53d2169ff Jun 05 13:17:38.512906 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.512863 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" event={"ID":"4273eacb-eadf-4bae-ba8d-d0f98f6096af","Type":"ContainerStarted","Data":"23e75e0b0ac38078bff83dca106d18dd4ca6c42a4a0f5024fa3df9a53d2169ff"} Jun 05 13:17:38.513202 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.513186 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:38.514123 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.514104 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:17:38.921427 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:38.921336 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:39.014923 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.014890 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:39.067015 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.066978 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-z7p24"] Jun 05 13:17:39.071766 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.071745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:17:39.074707 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.074689 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-z45qb\"" Jun 05 13:17:39.076525 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.076504 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z7p24"] Jun 05 13:17:39.198343 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.198231 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvz8f\" (UniqueName: \"kubernetes.io/projected/a49b3e6b-fdd3-48ce-9efe-8523199f37e2-kube-api-access-hvz8f\") pod \"authorino-75d556bd77-z7p24\" (UID: \"a49b3e6b-fdd3-48ce-9efe-8523199f37e2\") " pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:17:39.299886 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.299852 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvz8f\" (UniqueName: \"kubernetes.io/projected/a49b3e6b-fdd3-48ce-9efe-8523199f37e2-kube-api-access-hvz8f\") pod \"authorino-75d556bd77-z7p24\" (UID: \"a49b3e6b-fdd3-48ce-9efe-8523199f37e2\") " pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:17:39.313907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.313848 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvz8f\" (UniqueName: \"kubernetes.io/projected/a49b3e6b-fdd3-48ce-9efe-8523199f37e2-kube-api-access-hvz8f\") pod \"authorino-75d556bd77-z7p24\" (UID: \"a49b3e6b-fdd3-48ce-9efe-8523199f37e2\") " pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:17:39.382108 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.382069 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:17:39.556534 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:39.556506 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z7p24"] Jun 05 13:17:39.558895 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:17:39.558856 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda49b3e6b_fdd3_48ce_9efe_8523199f37e2.slice/crio-2ee1be8681097e6201e2ce0364fd30722feea901914907d7e38076500bbdfe42 WatchSource:0}: Error finding container 2ee1be8681097e6201e2ce0364fd30722feea901914907d7e38076500bbdfe42: Status 404 returned error can't find the container with id 2ee1be8681097e6201e2ce0364fd30722feea901914907d7e38076500bbdfe42 Jun 05 13:17:40.526811 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:40.526770 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z7p24" event={"ID":"a49b3e6b-fdd3-48ce-9efe-8523199f37e2","Type":"ContainerStarted","Data":"2ee1be8681097e6201e2ce0364fd30722feea901914907d7e38076500bbdfe42"} Jun 05 13:17:41.534435 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:41.534396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" event={"ID":"4273eacb-eadf-4bae-ba8d-d0f98f6096af","Type":"ContainerStarted","Data":"9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f"} Jun 05 13:17:41.534907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:41.534602 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:41.556769 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:41.556672 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" podStartSLOduration=0.754473586 podStartE2EDuration="3.556655047s" totalImagesPullingTime="2.802181461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:17:41.554534489 +0000 UTC m=+674.195858291" watchObservedRunningTime="2026-06-05 13:17:41.556655047 +0000 UTC m=+674.197978848" Jun 05 13:17:43.545586 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:43.545553 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z7p24" event={"ID":"a49b3e6b-fdd3-48ce-9efe-8523199f37e2","Type":"ContainerStarted","Data":"8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331"} Jun 05 13:17:43.563937 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:43.563883 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-z7p24" podStartSLOduration=0.994710577 podStartE2EDuration="4.563868692s" totalImagesPullingTime="3.569158115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:17:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:17:43.562806671 +0000 UTC m=+676.204130491" watchObservedRunningTime="2026-06-05 13:17:43.563868692 +0000 UTC m=+676.205192492" Jun 05 13:17:52.540816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:52.540778 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:52.724477 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:52.724437 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:52.724700 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:52.724671 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" podUID="4273eacb-eadf-4bae-ba8d-d0f98f6096af" containerName="limitador" containerID="cri-o://9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f" gracePeriod=30 Jun 05 13:17:53.275260 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.275222 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:53.334137 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.334056 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4273eacb-eadf-4bae-ba8d-d0f98f6096af-config-file\") pod \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " Jun 05 13:17:53.334137 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.334102 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hknv\" (UniqueName: \"kubernetes.io/projected/4273eacb-eadf-4bae-ba8d-d0f98f6096af-kube-api-access-5hknv\") pod \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\" (UID: \"4273eacb-eadf-4bae-ba8d-d0f98f6096af\") " Jun 05 13:17:53.334444 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.334420 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4273eacb-eadf-4bae-ba8d-d0f98f6096af-config-file" (OuterVolumeSpecName: "config-file") pod "4273eacb-eadf-4bae-ba8d-d0f98f6096af" (UID: "4273eacb-eadf-4bae-ba8d-d0f98f6096af"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 05 13:17:53.336144 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.336117 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4273eacb-eadf-4bae-ba8d-d0f98f6096af-kube-api-access-5hknv" (OuterVolumeSpecName: "kube-api-access-5hknv") pod "4273eacb-eadf-4bae-ba8d-d0f98f6096af" (UID: "4273eacb-eadf-4bae-ba8d-d0f98f6096af"). InnerVolumeSpecName "kube-api-access-5hknv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:17:53.435004 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.434958 2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4273eacb-eadf-4bae-ba8d-d0f98f6096af-config-file\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:17:53.435004 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.434996 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hknv\" (UniqueName: \"kubernetes.io/projected/4273eacb-eadf-4bae-ba8d-d0f98f6096af-kube-api-access-5hknv\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:17:53.592058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.591974 2570 generic.go:356] "Generic (PLEG): container finished" podID="4273eacb-eadf-4bae-ba8d-d0f98f6096af" containerID="9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f" exitCode=0 Jun 05 13:17:53.592058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.592018 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" Jun 05 13:17:53.592058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.592049 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" event={"ID":"4273eacb-eadf-4bae-ba8d-d0f98f6096af","Type":"ContainerDied","Data":"9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f"} Jun 05 13:17:53.592555 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.592087 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gf6sb" event={"ID":"4273eacb-eadf-4bae-ba8d-d0f98f6096af","Type":"ContainerDied","Data":"23e75e0b0ac38078bff83dca106d18dd4ca6c42a4a0f5024fa3df9a53d2169ff"} Jun 05 13:17:53.592555 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.592108 2570 scope.go:117] "RemoveContainer" containerID="9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f" Jun 05 13:17:53.601149 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.601134 2570 scope.go:117] "RemoveContainer" containerID="9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f" Jun 05 13:17:53.601441 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:17:53.601421 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f\": container with ID starting with 9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f not found: ID does not exist" containerID="9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f" Jun 05 13:17:53.601519 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.601446 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f"} err="failed to get container status \"9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f\": rpc error: code = NotFound desc = could not find container \"9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f\": container with ID starting with 9858f88a97f4d6015f22d8f1ed2e29ee0d5996e20b36e886b672e5f1d689867f not found: ID does not exist" Jun 05 13:17:53.614491 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.614466 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:53.620527 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.620508 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gf6sb"] Jun 05 13:17:53.782589 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:53.782558 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4273eacb-eadf-4bae-ba8d-d0f98f6096af" path="/var/lib/kubelet/pods/4273eacb-eadf-4bae-ba8d-d0f98f6096af/volumes" Jun 05 13:17:54.038691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.038659 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-6mdxb"] Jun 05 13:17:54.039038 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.039024 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4273eacb-eadf-4bae-ba8d-d0f98f6096af" containerName="limitador" Jun 05 13:17:54.039108 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.039039 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4273eacb-eadf-4bae-ba8d-d0f98f6096af" containerName="limitador" Jun 05 13:17:54.039166 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.039110 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4273eacb-eadf-4bae-ba8d-d0f98f6096af" containerName="limitador" Jun 05 13:17:54.043239 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.043220 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.046164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.046146 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 05 13:17:54.046343 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.046324 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6tdh5\"" Jun 05 13:17:54.050690 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.050669 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6mdxb"] Jun 05 13:17:54.071848 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.071815 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:17:54.142457 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.142423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/66bdc07f-e31b-40c9-972a-77919bd3187a-data\") pod \"postgres-69fb4594cf-6mdxb\" (UID: \"66bdc07f-e31b-40c9-972a-77919bd3187a\") " pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.142619 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.142505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmpzt\" (UniqueName: \"kubernetes.io/projected/66bdc07f-e31b-40c9-972a-77919bd3187a-kube-api-access-tmpzt\") pod \"postgres-69fb4594cf-6mdxb\" (UID: \"66bdc07f-e31b-40c9-972a-77919bd3187a\") " pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.243148 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.243109 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmpzt\" (UniqueName: \"kubernetes.io/projected/66bdc07f-e31b-40c9-972a-77919bd3187a-kube-api-access-tmpzt\") pod \"postgres-69fb4594cf-6mdxb\" (UID: \"66bdc07f-e31b-40c9-972a-77919bd3187a\") " pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.243334 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.243168 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/66bdc07f-e31b-40c9-972a-77919bd3187a-data\") pod \"postgres-69fb4594cf-6mdxb\" (UID: \"66bdc07f-e31b-40c9-972a-77919bd3187a\") " pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.243546 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.243530 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/66bdc07f-e31b-40c9-972a-77919bd3187a-data\") pod \"postgres-69fb4594cf-6mdxb\" (UID: \"66bdc07f-e31b-40c9-972a-77919bd3187a\") " pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.251901 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.251870 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmpzt\" (UniqueName: \"kubernetes.io/projected/66bdc07f-e31b-40c9-972a-77919bd3187a-kube-api-access-tmpzt\") pod \"postgres-69fb4594cf-6mdxb\" (UID: \"66bdc07f-e31b-40c9-972a-77919bd3187a\") " pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.357832 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.357756 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:54.488766 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.488741 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6mdxb"] Jun 05 13:17:54.490814 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:17:54.490785 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bdc07f_e31b_40c9_972a_77919bd3187a.slice/crio-c22b72cd9914406b66c20fe94d6f71ef7b454f80ce25b41e2991fac0bf0a381e WatchSource:0}: Error finding container c22b72cd9914406b66c20fe94d6f71ef7b454f80ce25b41e2991fac0bf0a381e: Status 404 returned error can't find the container with id c22b72cd9914406b66c20fe94d6f71ef7b454f80ce25b41e2991fac0bf0a381e Jun 05 13:17:54.598040 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:54.598002 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6mdxb" event={"ID":"66bdc07f-e31b-40c9-972a-77919bd3187a","Type":"ContainerStarted","Data":"c22b72cd9914406b66c20fe94d6f71ef7b454f80ce25b41e2991fac0bf0a381e"} Jun 05 13:17:58.619522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:58.619485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6mdxb" event={"ID":"66bdc07f-e31b-40c9-972a-77919bd3187a","Type":"ContainerStarted","Data":"2910e3d09581eb476267b1ba8fcc50b7025f2f30b344b04ac0d6431c1dc4c171"} Jun 05 13:17:58.619938 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:58.619634 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:17:58.641345 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:17:58.641291 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-6mdxb" podStartSLOduration=1.551802827 podStartE2EDuration="4.64127648s" totalImagesPullingTime="3.089473653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:17:58.639473046 +0000 UTC m=+691.280796848" watchObservedRunningTime="2026-06-05 13:17:58.64127648 +0000 UTC m=+691.282600281" Jun 05 13:18:04.675128 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:04.675090 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-6mdxb" Jun 05 13:18:05.038414 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:05.038381 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:05.117169 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:05.117139 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:05.266362 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:05.266327 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:29.190288 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.190226 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:29.285929 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.285895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:29.586802 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.586762 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:18:29.591492 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.591466 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:29.594334 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.594309 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 05 13:18:29.594476 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.594381 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 05 13:18:29.594476 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.594317 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 05 13:18:29.595339 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.595321 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-ggq5l\"" Jun 05 13:18:29.600771 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.600749 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:18:29.666130 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.666097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk7nw\" (UniqueName: \"kubernetes.io/projected/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75-kube-api-access-mk7nw\") pod \"maas-keycloak-0\" (UID: \"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:29.766766 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.766728 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk7nw\" (UniqueName: \"kubernetes.io/projected/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75-kube-api-access-mk7nw\") pod \"maas-keycloak-0\" (UID: \"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:29.776138 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.776109 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk7nw\" (UniqueName: \"kubernetes.io/projected/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75-kube-api-access-mk7nw\") pod \"maas-keycloak-0\" (UID: \"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:29.902456 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:29.902365 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:30.041322 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:30.041294 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:18:30.043991 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:18:30.043966 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc21f0821_1dae_47f2_b0ac_1c03d6bf2c75.slice/crio-8c79c1278e10de1b9abfbddce8fe450ba6c2893c7ca4beb9bcdb71f6f9648f4f WatchSource:0}: Error finding container 8c79c1278e10de1b9abfbddce8fe450ba6c2893c7ca4beb9bcdb71f6f9648f4f: Status 404 returned error can't find the container with id 8c79c1278e10de1b9abfbddce8fe450ba6c2893c7ca4beb9bcdb71f6f9648f4f Jun 05 13:18:30.777964 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:30.777926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75","Type":"ContainerStarted","Data":"8c79c1278e10de1b9abfbddce8fe450ba6c2893c7ca4beb9bcdb71f6f9648f4f"} Jun 05 13:18:35.805403 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:35.805359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75","Type":"ContainerStarted","Data":"66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167"} Jun 05 13:18:35.827360 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:35.827298 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.084941892 podStartE2EDuration="6.827277416s" totalImagesPullingTime="4.742335524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:18:35.825814746 +0000 UTC m=+728.467138548" watchObservedRunningTime="2026-06-05 13:18:35.827277416 +0000 UTC m=+728.468601217" Jun 05 13:18:35.902881 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:35.902842 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:35.904723 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:35.904687 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:36.903137 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:36.903084 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:37.903459 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:37.903236 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:38.703724 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:38.703672 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:38.803658 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:38.803610 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:18:38.903096 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:38.903043 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:39.903503 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:39.903454 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:39.904117 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:39.903819 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:40.902865 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:40.902815 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:41.903076 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:41.903024 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:42.903011 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:42.902963 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:43.905365 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:43.904680 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:44.902992 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:44.902942 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:45.903418 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:45.903370 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:46.903708 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:46.903648 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:47.903460 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:47.903411 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 05 13:18:48.997198 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:48.997160 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:49.012354 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:49.012306 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 05 13:18:59.003784 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:59.003745 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 05 13:18:59.721030 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:18:59.720989 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:00.246135 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.246098 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:00.489012 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.488978 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bdd567f84-6lbll"] Jun 05 13:19:00.492458 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.492437 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.495034 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.495015 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 05 13:19:00.501298 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.501276 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bdd567f84-6lbll"] Jun 05 13:19:00.585462 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.585426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffxz4\" (UniqueName: \"kubernetes.io/projected/c54cabff-0368-4dc1-9a1b-289cbe3d8778-kube-api-access-ffxz4\") pod \"authorino-5bdd567f84-6lbll\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.585655 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.585499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c54cabff-0368-4dc1-9a1b-289cbe3d8778-tls-cert\") pod \"authorino-5bdd567f84-6lbll\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.686368 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.686334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c54cabff-0368-4dc1-9a1b-289cbe3d8778-tls-cert\") pod \"authorino-5bdd567f84-6lbll\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.686536 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.686411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffxz4\" (UniqueName: \"kubernetes.io/projected/c54cabff-0368-4dc1-9a1b-289cbe3d8778-kube-api-access-ffxz4\") pod \"authorino-5bdd567f84-6lbll\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.688997 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.688970 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c54cabff-0368-4dc1-9a1b-289cbe3d8778-tls-cert\") pod \"authorino-5bdd567f84-6lbll\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.694877 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.694848 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffxz4\" (UniqueName: \"kubernetes.io/projected/c54cabff-0368-4dc1-9a1b-289cbe3d8778-kube-api-access-ffxz4\") pod \"authorino-5bdd567f84-6lbll\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.801841 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.801742 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:19:00.938348 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.938322 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bdd567f84-6lbll"] Jun 05 13:19:00.940151 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:00.940124 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc54cabff_0368_4dc1_9a1b_289cbe3d8778.slice/crio-4bc29a7284fd004a4c57be43f0cc4da68c442b7a2b5fbae530621bf4dc614f45 WatchSource:0}: Error finding container 4bc29a7284fd004a4c57be43f0cc4da68c442b7a2b5fbae530621bf4dc614f45: Status 404 returned error can't find the container with id 4bc29a7284fd004a4c57be43f0cc4da68c442b7a2b5fbae530621bf4dc614f45 Jun 05 13:19:00.955901 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:00.955877 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bdd567f84-6lbll" event={"ID":"c54cabff-0368-4dc1-9a1b-289cbe3d8778","Type":"ContainerStarted","Data":"4bc29a7284fd004a4c57be43f0cc4da68c442b7a2b5fbae530621bf4dc614f45"} Jun 05 13:19:01.962393 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:01.962354 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bdd567f84-6lbll" event={"ID":"c54cabff-0368-4dc1-9a1b-289cbe3d8778","Type":"ContainerStarted","Data":"6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4"} Jun 05 13:19:01.982355 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:01.982289 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bdd567f84-6lbll" podStartSLOduration=1.705972015 podStartE2EDuration="1.982240055s" totalImagesPullingTime="276.26804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:01.981375799 +0000 UTC m=+754.622699604" watchObservedRunningTime="2026-06-05 13:19:01.982240055 +0000 UTC m=+754.623563897" Jun 05 13:19:02.009214 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.009178 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z7p24"] Jun 05 13:19:02.009434 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.009402 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-z7p24" podUID="a49b3e6b-fdd3-48ce-9efe-8523199f37e2" containerName="authorino" containerID="cri-o://8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331" gracePeriod=30 Jun 05 13:19:02.260384 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.260357 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:19:02.403792 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.403761 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvz8f\" (UniqueName: \"kubernetes.io/projected/a49b3e6b-fdd3-48ce-9efe-8523199f37e2-kube-api-access-hvz8f\") pod \"a49b3e6b-fdd3-48ce-9efe-8523199f37e2\" (UID: \"a49b3e6b-fdd3-48ce-9efe-8523199f37e2\") " Jun 05 13:19:02.405786 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.405751 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49b3e6b-fdd3-48ce-9efe-8523199f37e2-kube-api-access-hvz8f" (OuterVolumeSpecName: "kube-api-access-hvz8f") pod "a49b3e6b-fdd3-48ce-9efe-8523199f37e2" (UID: "a49b3e6b-fdd3-48ce-9efe-8523199f37e2"). InnerVolumeSpecName "kube-api-access-hvz8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:19:02.505265 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.505157 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvz8f\" (UniqueName: \"kubernetes.io/projected/a49b3e6b-fdd3-48ce-9efe-8523199f37e2-kube-api-access-hvz8f\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:19:02.968736 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.968655 2570 generic.go:356] "Generic (PLEG): container finished" podID="a49b3e6b-fdd3-48ce-9efe-8523199f37e2" containerID="8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331" exitCode=0 Jun 05 13:19:02.968736 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.968695 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z7p24" Jun 05 13:19:02.969222 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.968739 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z7p24" event={"ID":"a49b3e6b-fdd3-48ce-9efe-8523199f37e2","Type":"ContainerDied","Data":"8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331"} Jun 05 13:19:02.969222 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.968776 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z7p24" event={"ID":"a49b3e6b-fdd3-48ce-9efe-8523199f37e2","Type":"ContainerDied","Data":"2ee1be8681097e6201e2ce0364fd30722feea901914907d7e38076500bbdfe42"} Jun 05 13:19:02.969222 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.968795 2570 scope.go:117] "RemoveContainer" containerID="8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331" Jun 05 13:19:02.977798 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.977780 2570 scope.go:117] "RemoveContainer" containerID="8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331" Jun 05 13:19:02.978070 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:19:02.978049 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331\": container with ID starting with 8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331 not found: ID does not exist" containerID="8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331" Jun 05 13:19:02.978164 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.978077 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331"} err="failed to get container status \"8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331\": rpc error: code = NotFound desc = could not find container \"8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331\": container with ID starting with 8c2f9db9b1ddc0a282d38b956e9a5132ff32ae1911ca20f8f78de9327935f331 not found: ID does not exist" Jun 05 13:19:02.990015 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.989985 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z7p24"] Jun 05 13:19:02.994825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:02.994799 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z7p24"] Jun 05 13:19:03.782584 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:03.782553 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49b3e6b-fdd3-48ce-9efe-8523199f37e2" path="/var/lib/kubelet/pods/a49b3e6b-fdd3-48ce-9efe-8523199f37e2/volumes" Jun 05 13:19:05.563958 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.563925 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d8b978455-scvbt"] Jun 05 13:19:05.564368 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.564353 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a49b3e6b-fdd3-48ce-9efe-8523199f37e2" containerName="authorino" Jun 05 13:19:05.564439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.564369 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a49b3e6b-fdd3-48ce-9efe-8523199f37e2" containerName="authorino" Jun 05 13:19:05.564439 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.564428 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a49b3e6b-fdd3-48ce-9efe-8523199f37e2" containerName="authorino" Jun 05 13:19:05.569499 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.569481 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:05.572597 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.572581 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-k7nrh\"" Jun 05 13:19:05.573021 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.573008 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 05 13:19:05.579213 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.579186 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8b978455-scvbt"] Jun 05 13:19:05.636491 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.636455 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wz6r\" (UniqueName: \"kubernetes.io/projected/8db0d792-d6ed-4930-8d87-3399731f0197-kube-api-access-9wz6r\") pod \"maas-controller-7d8b978455-scvbt\" (UID: \"8db0d792-d6ed-4930-8d87-3399731f0197\") " pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:05.716003 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.715967 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58ccf9c474-2rn5l"] Jun 05 13:19:05.719681 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.719659 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:05.728412 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.728391 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-2rn5l"] Jun 05 13:19:05.736938 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.736915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz6r\" (UniqueName: \"kubernetes.io/projected/8db0d792-d6ed-4930-8d87-3399731f0197-kube-api-access-9wz6r\") pod \"maas-controller-7d8b978455-scvbt\" (UID: \"8db0d792-d6ed-4930-8d87-3399731f0197\") " pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:05.737051 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.736978 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4r27\" (UniqueName: \"kubernetes.io/projected/25504245-5107-420f-8485-b58484beb2d2-kube-api-access-v4r27\") pod \"maas-controller-58ccf9c474-2rn5l\" (UID: \"25504245-5107-420f-8485-b58484beb2d2\") " pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:05.745677 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.745654 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wz6r\" (UniqueName: \"kubernetes.io/projected/8db0d792-d6ed-4930-8d87-3399731f0197-kube-api-access-9wz6r\") pod \"maas-controller-7d8b978455-scvbt\" (UID: \"8db0d792-d6ed-4930-8d87-3399731f0197\") " pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:05.837787 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.837696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4r27\" (UniqueName: \"kubernetes.io/projected/25504245-5107-420f-8485-b58484beb2d2-kube-api-access-v4r27\") pod \"maas-controller-58ccf9c474-2rn5l\" (UID: \"25504245-5107-420f-8485-b58484beb2d2\") " pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:05.847995 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.847960 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4r27\" (UniqueName: \"kubernetes.io/projected/25504245-5107-420f-8485-b58484beb2d2-kube-api-access-v4r27\") pod \"maas-controller-58ccf9c474-2rn5l\" (UID: \"25504245-5107-420f-8485-b58484beb2d2\") " pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:05.880666 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:05.880629 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:06.018616 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:06.018586 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8b978455-scvbt"] Jun 05 13:19:06.020632 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:06.020593 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db0d792_d6ed_4930_8d87_3399731f0197.slice/crio-9d585bd83cad5437343a4f692959d0fb3f8a92153a537240005f42251e9f92df WatchSource:0}: Error finding container 9d585bd83cad5437343a4f692959d0fb3f8a92153a537240005f42251e9f92df: Status 404 returned error can't find the container with id 9d585bd83cad5437343a4f692959d0fb3f8a92153a537240005f42251e9f92df Jun 05 13:19:06.022143 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:06.022125 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 05 13:19:06.031917 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:06.031895 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:06.173099 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:06.173070 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-2rn5l"] Jun 05 13:19:06.176492 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:06.176461 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25504245_5107_420f_8485_b58484beb2d2.slice/crio-d653eec2cc18e199f766edd14e347bb99de3372e8ce8b39bd04bbf836967272e WatchSource:0}: Error finding container d653eec2cc18e199f766edd14e347bb99de3372e8ce8b39bd04bbf836967272e: Status 404 returned error can't find the container with id d653eec2cc18e199f766edd14e347bb99de3372e8ce8b39bd04bbf836967272e Jun 05 13:19:07.002058 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:07.001967 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8b978455-scvbt" event={"ID":"8db0d792-d6ed-4930-8d87-3399731f0197","Type":"ContainerStarted","Data":"9d585bd83cad5437343a4f692959d0fb3f8a92153a537240005f42251e9f92df"} Jun 05 13:19:07.004072 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:07.003993 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" event={"ID":"25504245-5107-420f-8485-b58484beb2d2","Type":"ContainerStarted","Data":"d653eec2cc18e199f766edd14e347bb99de3372e8ce8b39bd04bbf836967272e"} Jun 05 13:19:09.522897 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:09.522849 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:09.635050 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:09.635021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:10.020866 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:10.020832 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8b978455-scvbt" event={"ID":"8db0d792-d6ed-4930-8d87-3399731f0197","Type":"ContainerStarted","Data":"f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364"} Jun 05 13:19:10.021076 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:10.021049 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:10.022300 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:10.022273 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" event={"ID":"25504245-5107-420f-8485-b58484beb2d2","Type":"ContainerStarted","Data":"53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349"} Jun 05 13:19:10.022410 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:10.022381 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:10.047003 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:10.046958 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d8b978455-scvbt" podStartSLOduration=1.809833738 podStartE2EDuration="5.046943995s" totalImagesPullingTime="3.237110257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:10.045235757 +0000 UTC m=+762.686559558" watchObservedRunningTime="2026-06-05 13:19:10.046943995 +0000 UTC m=+762.688267795" Jun 05 13:19:10.064337 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:10.064284 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" podStartSLOduration=1.975058562 podStartE2EDuration="5.064269651s" totalImagesPullingTime="3.089211089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:10.062576633 +0000 UTC m=+762.703900436" watchObservedRunningTime="2026-06-05 13:19:10.064269651 +0000 UTC m=+762.705593479" Jun 05 13:19:21.035009 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.034975 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:21.035419 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.035186 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:19:21.097614 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.097584 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8b978455-scvbt"] Jun 05 13:19:21.097825 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.097781 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7d8b978455-scvbt" podUID="8db0d792-d6ed-4930-8d87-3399731f0197" containerName="manager" containerID="cri-o://f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364" gracePeriod=30 Jun 05 13:19:21.342156 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.342130 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:21.380649 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.380609 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wz6r\" (UniqueName: \"kubernetes.io/projected/8db0d792-d6ed-4930-8d87-3399731f0197-kube-api-access-9wz6r\") pod \"8db0d792-d6ed-4930-8d87-3399731f0197\" (UID: \"8db0d792-d6ed-4930-8d87-3399731f0197\") " Jun 05 13:19:21.382696 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.382670 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db0d792-d6ed-4930-8d87-3399731f0197-kube-api-access-9wz6r" (OuterVolumeSpecName: "kube-api-access-9wz6r") pod "8db0d792-d6ed-4930-8d87-3399731f0197" (UID: "8db0d792-d6ed-4930-8d87-3399731f0197"). InnerVolumeSpecName "kube-api-access-9wz6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:19:21.481770 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:21.481728 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wz6r\" (UniqueName: \"kubernetes.io/projected/8db0d792-d6ed-4930-8d87-3399731f0197-kube-api-access-9wz6r\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:19:22.082463 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.082362 2570 generic.go:356] "Generic (PLEG): container finished" podID="8db0d792-d6ed-4930-8d87-3399731f0197" containerID="f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364" exitCode=0 Jun 05 13:19:22.082463 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.082400 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8b978455-scvbt" Jun 05 13:19:22.082463 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.082442 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8b978455-scvbt" event={"ID":"8db0d792-d6ed-4930-8d87-3399731f0197","Type":"ContainerDied","Data":"f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364"} Jun 05 13:19:22.082983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.082496 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8b978455-scvbt" event={"ID":"8db0d792-d6ed-4930-8d87-3399731f0197","Type":"ContainerDied","Data":"9d585bd83cad5437343a4f692959d0fb3f8a92153a537240005f42251e9f92df"} Jun 05 13:19:22.082983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.082523 2570 scope.go:117] "RemoveContainer" containerID="f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364" Jun 05 13:19:22.090962 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.090947 2570 scope.go:117] "RemoveContainer" containerID="f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364" Jun 05 13:19:22.091223 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:19:22.091199 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364\": container with ID starting with f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364 not found: ID does not exist" containerID="f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364" Jun 05 13:19:22.091345 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.091227 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364"} err="failed to get container status \"f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364\": rpc error: code = NotFound desc = could not find container \"f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364\": container with ID starting with f2416edb55b59a152feade3639ffee4ade9780c81ff0368ed5aad2311b893364 not found: ID does not exist" Jun 05 13:19:22.100161 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.100136 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8b978455-scvbt"] Jun 05 13:19:22.104091 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:22.104068 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d8b978455-scvbt"] Jun 05 13:19:23.782475 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:23.782441 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db0d792-d6ed-4930-8d87-3399731f0197" path="/var/lib/kubelet/pods/8db0d792-d6ed-4930-8d87-3399731f0197/volumes" Jun 05 13:19:40.621473 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.621433 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:40.680554 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.680522 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-56fcb9bb9-5mmfz"] Jun 05 13:19:40.680891 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.680879 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db0d792-d6ed-4930-8d87-3399731f0197" containerName="manager" Jun 05 13:19:40.680953 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.680892 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db0d792-d6ed-4930-8d87-3399731f0197" containerName="manager" Jun 05 13:19:40.680999 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.680987 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db0d792-d6ed-4930-8d87-3399731f0197" containerName="manager" Jun 05 13:19:40.731929 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.731884 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-56fcb9bb9-5mmfz"] Jun 05 13:19:40.732177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.731959 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:40.732177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.732090 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:40.734918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.734899 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hgh45\"" Jun 05 13:19:40.735759 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.735743 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 05 13:19:40.861784 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.861744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdkjn\" (UniqueName: \"kubernetes.io/projected/6e88bf51-725e-458e-91e1-9d9d06b83cfc-kube-api-access-hdkjn\") pod \"maas-api-56fcb9bb9-5mmfz\" (UID: \"6e88bf51-725e-458e-91e1-9d9d06b83cfc\") " pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:40.861971 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.861829 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e88bf51-725e-458e-91e1-9d9d06b83cfc-maas-api-tls\") pod \"maas-api-56fcb9bb9-5mmfz\" (UID: \"6e88bf51-725e-458e-91e1-9d9d06b83cfc\") " pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:40.963283 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.963178 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdkjn\" (UniqueName: \"kubernetes.io/projected/6e88bf51-725e-458e-91e1-9d9d06b83cfc-kube-api-access-hdkjn\") pod \"maas-api-56fcb9bb9-5mmfz\" (UID: \"6e88bf51-725e-458e-91e1-9d9d06b83cfc\") " pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:40.963454 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.963362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e88bf51-725e-458e-91e1-9d9d06b83cfc-maas-api-tls\") pod \"maas-api-56fcb9bb9-5mmfz\" (UID: \"6e88bf51-725e-458e-91e1-9d9d06b83cfc\") " pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:40.965875 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.965852 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e88bf51-725e-458e-91e1-9d9d06b83cfc-maas-api-tls\") pod \"maas-api-56fcb9bb9-5mmfz\" (UID: \"6e88bf51-725e-458e-91e1-9d9d06b83cfc\") " pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:40.971908 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:40.971885 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdkjn\" (UniqueName: \"kubernetes.io/projected/6e88bf51-725e-458e-91e1-9d9d06b83cfc-kube-api-access-hdkjn\") pod \"maas-api-56fcb9bb9-5mmfz\" (UID: \"6e88bf51-725e-458e-91e1-9d9d06b83cfc\") " pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:41.042199 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.042171 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:41.057192 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.057164 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:41.155617 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.155587 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:41.164326 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.164297 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-nm7nl"] Jun 05 13:19:41.167473 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.167450 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:41.170069 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.170051 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 05 13:19:41.179598 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.179575 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-nm7nl"] Jun 05 13:19:41.189197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.189171 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-gc4zb"] Jun 05 13:19:41.191918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.191552 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:41.194191 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.194174 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9cm28\"" Jun 05 13:19:41.194857 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.194833 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-56fcb9bb9-5mmfz"] Jun 05 13:19:41.197226 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:41.197202 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e88bf51_725e_458e_91e1_9d9d06b83cfc.slice/crio-9ba153b3e1e2a80d3e9e3feae29500294994f08f35113b06ccd42bd38a4495cb WatchSource:0}: Error finding container 9ba153b3e1e2a80d3e9e3feae29500294994f08f35113b06ccd42bd38a4495cb: Status 404 returned error can't find the container with id 9ba153b3e1e2a80d3e9e3feae29500294994f08f35113b06ccd42bd38a4495cb Jun 05 13:19:41.200427 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.200410 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-gc4zb"] Jun 05 13:19:41.266844 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.266811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7868l\" (UniqueName: \"kubernetes.io/projected/60724d9d-0330-45d1-9d8d-e5e756f29a60-kube-api-access-7868l\") pod \"payload-pre-processing-67d7689b65-nm7nl\" (UID: \"60724d9d-0330-45d1-9d8d-e5e756f29a60\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:41.367816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.367785 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7868l\" (UniqueName: \"kubernetes.io/projected/60724d9d-0330-45d1-9d8d-e5e756f29a60-kube-api-access-7868l\") pod \"payload-pre-processing-67d7689b65-nm7nl\" (UID: \"60724d9d-0330-45d1-9d8d-e5e756f29a60\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:41.367983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.367827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gjbp\" (UniqueName: \"kubernetes.io/projected/33bbb2ca-3289-4f75-b469-5776d125a755-kube-api-access-9gjbp\") pod \"payload-processing-7c6c859784-gc4zb\" (UID: \"33bbb2ca-3289-4f75-b469-5776d125a755\") " pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:41.380622 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.380594 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7868l\" (UniqueName: \"kubernetes.io/projected/60724d9d-0330-45d1-9d8d-e5e756f29a60-kube-api-access-7868l\") pod \"payload-pre-processing-67d7689b65-nm7nl\" (UID: \"60724d9d-0330-45d1-9d8d-e5e756f29a60\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:41.468555 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.468518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gjbp\" (UniqueName: \"kubernetes.io/projected/33bbb2ca-3289-4f75-b469-5776d125a755-kube-api-access-9gjbp\") pod \"payload-processing-7c6c859784-gc4zb\" (UID: \"33bbb2ca-3289-4f75-b469-5776d125a755\") " pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:41.477368 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.477344 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gjbp\" (UniqueName: \"kubernetes.io/projected/33bbb2ca-3289-4f75-b469-5776d125a755-kube-api-access-9gjbp\") pod \"payload-processing-7c6c859784-gc4zb\" (UID: \"33bbb2ca-3289-4f75-b469-5776d125a755\") " pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:41.479019 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.478998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:41.512049 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.512022 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:41.625742 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.625718 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-nm7nl"] Jun 05 13:19:41.628690 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:41.628660 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60724d9d_0330_45d1_9d8d_e5e756f29a60.slice/crio-6f71f235f60c14334fc4d7968885e4ae5ba4d2ab9c6ecafd195c4a20c8ca784b WatchSource:0}: Error finding container 6f71f235f60c14334fc4d7968885e4ae5ba4d2ab9c6ecafd195c4a20c8ca784b: Status 404 returned error can't find the container with id 6f71f235f60c14334fc4d7968885e4ae5ba4d2ab9c6ecafd195c4a20c8ca784b Jun 05 13:19:41.664038 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:41.664014 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-gc4zb"] Jun 05 13:19:41.665588 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:41.665556 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bbb2ca_3289_4f75_b469_5776d125a755.slice/crio-cf70128739293d0fa02fd26b707e72c45cc0c764f1070e9e749fa0056aa41360 WatchSource:0}: Error finding container cf70128739293d0fa02fd26b707e72c45cc0c764f1070e9e749fa0056aa41360: Status 404 returned error can't find the container with id cf70128739293d0fa02fd26b707e72c45cc0c764f1070e9e749fa0056aa41360 Jun 05 13:19:42.181010 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:42.180977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" event={"ID":"33bbb2ca-3289-4f75-b469-5776d125a755","Type":"ContainerStarted","Data":"cf70128739293d0fa02fd26b707e72c45cc0c764f1070e9e749fa0056aa41360"} Jun 05 13:19:42.182454 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:42.182425 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" event={"ID":"60724d9d-0330-45d1-9d8d-e5e756f29a60","Type":"ContainerStarted","Data":"6f71f235f60c14334fc4d7968885e4ae5ba4d2ab9c6ecafd195c4a20c8ca784b"} Jun 05 13:19:42.183762 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:42.183737 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" event={"ID":"6e88bf51-725e-458e-91e1-9d9d06b83cfc","Type":"ContainerStarted","Data":"9ba153b3e1e2a80d3e9e3feae29500294994f08f35113b06ccd42bd38a4495cb"} Jun 05 13:19:44.198677 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:44.198644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" event={"ID":"6e88bf51-725e-458e-91e1-9d9d06b83cfc","Type":"ContainerStarted","Data":"ad2372ca716241f7a082cb9ec4a22fb2816db34d71600868a3056a0eb69758e5"} Jun 05 13:19:44.199147 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:44.198716 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:45.205196 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.205099 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" event={"ID":"33bbb2ca-3289-4f75-b469-5776d125a755","Type":"ContainerStarted","Data":"9d59ec803fecbcc2aac7da67726f58c3b8fa9c39405cd04d8831a20628c92da4"} Jun 05 13:19:45.205627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.205205 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:45.206840 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.206815 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" event={"ID":"60724d9d-0330-45d1-9d8d-e5e756f29a60","Type":"ContainerStarted","Data":"4569cd96fa23f687af89b0d4a339fa623c755dced4bdf2e4b52822efb9b3863c"} Jun 05 13:19:45.206984 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.206874 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:45.221956 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.221895 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" podStartSLOduration=0.945665984 podStartE2EDuration="4.221798647s" totalImagesPullingTime="3.276132663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:45.221222483 +0000 UTC m=+797.862546287" watchObservedRunningTime="2026-06-05 13:19:45.221798647 +0000 UTC m=+797.863122449" Jun 05 13:19:45.222941 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.222895 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" podStartSLOduration=2.971495889 podStartE2EDuration="5.222866956s" totalImagesPullingTime="2.251371067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:44.235537677 +0000 UTC m=+796.876861479" watchObservedRunningTime="2026-06-05 13:19:45.222866956 +0000 UTC m=+797.864190773" Jun 05 13:19:45.239858 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:45.239802 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" podStartSLOduration=0.934343598 podStartE2EDuration="4.239781913s" totalImagesPullingTime="3.305438315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:45.237721105 +0000 UTC m=+797.879044909" watchObservedRunningTime="2026-06-05 13:19:45.239781913 +0000 UTC m=+797.881105716" Jun 05 13:19:49.228403 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:49.228374 2570 generic.go:356] "Generic (PLEG): container finished" podID="8163f9c8-dcf3-4d8e-807d-0b40b2b87048" containerID="21bda51947740c81dfb974282360bc14077a4b73ddc4e1cfbb5d1b92f89f4274" exitCode=0 Jun 05 13:19:49.228795 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:49.228433 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" event={"ID":"8163f9c8-dcf3-4d8e-807d-0b40b2b87048","Type":"ContainerDied","Data":"21bda51947740c81dfb974282360bc14077a4b73ddc4e1cfbb5d1b92f89f4274"} Jun 05 13:19:49.228795 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:49.228791 2570 scope.go:117] "RemoveContainer" containerID="21bda51947740c81dfb974282360bc14077a4b73ddc4e1cfbb5d1b92f89f4274" Jun 05 13:19:49.229808 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:49.229782 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 05 13:19:49.229883 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:49.229847 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 05 13:19:49.229883 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:49.229876 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 05 13:19:50.214594 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:50.214565 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-56fcb9bb9-5mmfz" Jun 05 13:19:50.254598 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:50.254563 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" event={"ID":"8163f9c8-dcf3-4d8e-807d-0b40b2b87048","Type":"ContainerStarted","Data":"a548b16edf85a3d2b1a007a302f0244a4586bf5c76f7e2ee8efa2544552f3577"} Jun 05 13:19:51.054085 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:51.054043 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:19:51.055350 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:51.055326 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zxjrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 05 13:19:51.055447 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:51.055373 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" podUID="8163f9c8-dcf3-4d8e-807d-0b40b2b87048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 05 13:19:51.959098 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:51.959065 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:19:51.959526 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:51.959297 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" containerID="cri-o://66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167" gracePeriod=30 Jun 05 13:19:52.054585 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:52.054544 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zxjrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 05 13:19:52.054764 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:52.054608 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" podUID="8163f9c8-dcf3-4d8e-807d-0b40b2b87048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 05 13:19:53.054179 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.054146 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zxjrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 05 13:19:53.054587 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.054200 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" podUID="8163f9c8-dcf3-4d8e-807d-0b40b2b87048" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 05 13:19:53.517271 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.516350 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:19:53.647213 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.647094 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:53.678132 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.678104 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mk7nw\" (UniqueName: \"kubernetes.io/projected/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75-kube-api-access-mk7nw\") pod \"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75\" (UID: \"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75\") " Jun 05 13:19:53.682152 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.682121 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75-kube-api-access-mk7nw" (OuterVolumeSpecName: "kube-api-access-mk7nw") pod "c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" (UID: "c21f0821-1dae-47f2-b0ac-1c03d6bf2c75"). InnerVolumeSpecName "kube-api-access-mk7nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:19:53.778842 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:53.778809 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk7nw\" (UniqueName: \"kubernetes.io/projected/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75-kube-api-access-mk7nw\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:19:54.145031 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.144997 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:19:54.277506 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.277473 2570 generic.go:356] "Generic (PLEG): container finished" podID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerID="66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167" exitCode=143 Jun 05 13:19:54.277672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.277521 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.277672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.277562 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75","Type":"ContainerDied","Data":"66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167"} Jun 05 13:19:54.277672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.277614 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"c21f0821-1dae-47f2-b0ac-1c03d6bf2c75","Type":"ContainerDied","Data":"8c79c1278e10de1b9abfbddce8fe450ba6c2893c7ca4beb9bcdb71f6f9648f4f"} Jun 05 13:19:54.277672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.277635 2570 scope.go:117] "RemoveContainer" containerID="66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167" Jun 05 13:19:54.279197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.279173 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zxjrd" Jun 05 13:19:54.287670 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.287653 2570 scope.go:117] "RemoveContainer" containerID="66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167" Jun 05 13:19:54.287963 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:19:54.287945 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167\": container with ID starting with 66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167 not found: ID does not exist" containerID="66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167" Jun 05 13:19:54.288051 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.287971 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167"} err="failed to get container status \"66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167\": rpc error: code = NotFound desc = could not find container \"66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167\": container with ID starting with 66a17210ed9c5352c6dc378c3421e05e05f84250e70001ba12a6a1049d2da167 not found: ID does not exist" Jun 05 13:19:54.322208 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.322174 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:19:54.328749 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.328719 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:19:54.351594 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.351569 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:19:54.352045 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.352021 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" Jun 05 13:19:54.352045 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.352040 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" Jun 05 13:19:54.352180 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.352100 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" containerName="keycloak" Jun 05 13:19:54.355177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.355152 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.357988 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.357797 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 05 13:19:54.357988 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.357800 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 05 13:19:54.357988 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.357866 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-ggq5l\"" Jun 05 13:19:54.357988 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.357898 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 05 13:19:54.358266 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.358192 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 05 13:19:54.366047 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.365265 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:19:54.384215 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.384148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbk2\" (UniqueName: \"kubernetes.io/projected/ad8a0a3c-4455-4910-b57f-f559e0670b0b-kube-api-access-5fbk2\") pod \"maas-keycloak-0\" (UID: \"ad8a0a3c-4455-4910-b57f-f559e0670b0b\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.384331 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.384239 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/ad8a0a3c-4455-4910-b57f-f559e0670b0b-test-realms\") pod \"maas-keycloak-0\" (UID: \"ad8a0a3c-4455-4910-b57f-f559e0670b0b\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.485629 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.485592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/ad8a0a3c-4455-4910-b57f-f559e0670b0b-test-realms\") pod \"maas-keycloak-0\" (UID: \"ad8a0a3c-4455-4910-b57f-f559e0670b0b\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.485776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.485642 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbk2\" (UniqueName: \"kubernetes.io/projected/ad8a0a3c-4455-4910-b57f-f559e0670b0b-kube-api-access-5fbk2\") pod \"maas-keycloak-0\" (UID: \"ad8a0a3c-4455-4910-b57f-f559e0670b0b\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.486318 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.486301 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/ad8a0a3c-4455-4910-b57f-f559e0670b0b-test-realms\") pod \"maas-keycloak-0\" (UID: \"ad8a0a3c-4455-4910-b57f-f559e0670b0b\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.494376 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.494346 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbk2\" (UniqueName: \"kubernetes.io/projected/ad8a0a3c-4455-4910-b57f-f559e0670b0b-kube-api-access-5fbk2\") pod \"maas-keycloak-0\" (UID: \"ad8a0a3c-4455-4910-b57f-f559e0670b0b\") " pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.667230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.667200 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:54.798359 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.798300 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 05 13:19:54.800847 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:19:54.800818 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8a0a3c_4455_4910_b57f_f559e0670b0b.slice/crio-822ead51eb67bce02828d2b6421be08743d96262c912b15ee0903c852a2420f9 WatchSource:0}: Error finding container 822ead51eb67bce02828d2b6421be08743d96262c912b15ee0903c852a2420f9: Status 404 returned error can't find the container with id 822ead51eb67bce02828d2b6421be08743d96262c912b15ee0903c852a2420f9 Jun 05 13:19:54.817139 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.817115 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:54.918427 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:54.918394 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:19:55.285648 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:55.285598 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"ad8a0a3c-4455-4910-b57f-f559e0670b0b","Type":"ContainerStarted","Data":"d1d8ec8183edf4207e2cb3a3b5c1028142198b5bcd0cd23f13d844f0586e5289"} Jun 05 13:19:55.286077 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:55.285723 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"ad8a0a3c-4455-4910-b57f-f559e0670b0b","Type":"ContainerStarted","Data":"822ead51eb67bce02828d2b6421be08743d96262c912b15ee0903c852a2420f9"} Jun 05 13:19:55.306301 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:55.306230 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.992931416 podStartE2EDuration="1.306212858s" totalImagesPullingTime="313.281442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:19:55.305084374 +0000 UTC m=+807.946408192" watchObservedRunningTime="2026-06-05 13:19:55.306212858 +0000 UTC m=+807.947536659" Jun 05 13:19:55.668312 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:55.668271 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 05 13:19:55.669914 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:55.669873 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:19:55.784561 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:55.784528 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c21f0821-1dae-47f2-b0ac-1c03d6bf2c75" path="/var/lib/kubelet/pods/c21f0821-1dae-47f2-b0ac-1c03d6bf2c75/volumes" Jun 05 13:19:56.214330 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:56.214300 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-gc4zb" Jun 05 13:19:56.214668 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:56.214648 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-nm7nl" Jun 05 13:19:56.668012 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:56.667962 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:19:57.668140 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:57.668092 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:19:58.668382 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:58.668331 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:19:59.667851 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:19:59.667806 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:00.668139 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:00.668089 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:01.668315 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:01.668268 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:02.668704 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:02.668654 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:03.668135 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:03.668081 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:04.667551 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:04.667517 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 05 13:20:04.668029 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:04.667884 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:05.668135 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:05.668082 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:06.668365 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:06.668322 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:07.668602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:07.668551 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 05 13:20:08.797545 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:08.797224 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 05 13:20:08.817337 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:08.816763 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="ad8a0a3c-4455-4910-b57f-f559e0670b0b" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 05 13:20:18.803511 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:18.803475 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 05 13:20:31.099983 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.099943 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bdd567f84-6lbll"] Jun 05 13:20:31.100457 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.100150 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5bdd567f84-6lbll" podUID="c54cabff-0368-4dc1-9a1b-289cbe3d8778" containerName="authorino" containerID="cri-o://6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4" gracePeriod=30 Jun 05 13:20:31.364132 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.364066 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:20:31.452602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.452565 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffxz4\" (UniqueName: \"kubernetes.io/projected/c54cabff-0368-4dc1-9a1b-289cbe3d8778-kube-api-access-ffxz4\") pod \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " Jun 05 13:20:31.452602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.452602 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c54cabff-0368-4dc1-9a1b-289cbe3d8778-tls-cert\") pod \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\" (UID: \"c54cabff-0368-4dc1-9a1b-289cbe3d8778\") " Jun 05 13:20:31.454671 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.454644 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54cabff-0368-4dc1-9a1b-289cbe3d8778-kube-api-access-ffxz4" (OuterVolumeSpecName: "kube-api-access-ffxz4") pod "c54cabff-0368-4dc1-9a1b-289cbe3d8778" (UID: "c54cabff-0368-4dc1-9a1b-289cbe3d8778"). InnerVolumeSpecName "kube-api-access-ffxz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:20:31.463177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.463147 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54cabff-0368-4dc1-9a1b-289cbe3d8778-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c54cabff-0368-4dc1-9a1b-289cbe3d8778" (UID: "c54cabff-0368-4dc1-9a1b-289cbe3d8778"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 05 13:20:31.508672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.508630 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54cabff-0368-4dc1-9a1b-289cbe3d8778" containerID="6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4" exitCode=0 Jun 05 13:20:31.508862 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.508687 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bdd567f84-6lbll" Jun 05 13:20:31.508862 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.508764 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bdd567f84-6lbll" event={"ID":"c54cabff-0368-4dc1-9a1b-289cbe3d8778","Type":"ContainerDied","Data":"6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4"} Jun 05 13:20:31.508862 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.508810 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bdd567f84-6lbll" event={"ID":"c54cabff-0368-4dc1-9a1b-289cbe3d8778","Type":"ContainerDied","Data":"4bc29a7284fd004a4c57be43f0cc4da68c442b7a2b5fbae530621bf4dc614f45"} Jun 05 13:20:31.508862 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.508834 2570 scope.go:117] "RemoveContainer" containerID="6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4" Jun 05 13:20:31.514541 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.514517 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:20:31.518336 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.518309 2570 scope.go:117] "RemoveContainer" containerID="6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4" Jun 05 13:20:31.518576 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:20:31.518556 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4\": container with ID starting with 6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4 not found: ID does not exist" containerID="6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4" Jun 05 13:20:31.518628 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.518584 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4"} err="failed to get container status \"6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4\": rpc error: code = NotFound desc = could not find container \"6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4\": container with ID starting with 6c1b1cde9b7f72d16a5a74be41165b78395e316f4f2974fbcaef3cffe17573c4 not found: ID does not exist" Jun 05 13:20:31.536935 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.536896 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bdd567f84-6lbll"] Jun 05 13:20:31.542801 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.542776 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bdd567f84-6lbll"] Jun 05 13:20:31.553305 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.553277 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffxz4\" (UniqueName: \"kubernetes.io/projected/c54cabff-0368-4dc1-9a1b-289cbe3d8778-kube-api-access-ffxz4\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:20:31.553305 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.553307 2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c54cabff-0368-4dc1-9a1b-289cbe3d8778-tls-cert\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:20:31.783060 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:31.783027 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54cabff-0368-4dc1-9a1b-289cbe3d8778" path="/var/lib/kubelet/pods/c54cabff-0368-4dc1-9a1b-289cbe3d8778/volumes" Jun 05 13:20:32.670717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:32.670686 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:20:33.342471 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:33.342437 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:20:35.836477 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:35.836442 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:20:49.589729 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.589691 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g"] Jun 05 13:20:49.590339 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.590320 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54cabff-0368-4dc1-9a1b-289cbe3d8778" containerName="authorino" Jun 05 13:20:49.590393 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.590342 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54cabff-0368-4dc1-9a1b-289cbe3d8778" containerName="authorino" Jun 05 13:20:49.590447 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.590435 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54cabff-0368-4dc1-9a1b-289cbe3d8778" containerName="authorino" Jun 05 13:20:49.595231 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.595211 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.598303 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.598283 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 05 13:20:49.598437 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.598355 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g52bz\"" Jun 05 13:20:49.599292 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.599259 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 05 13:20:49.599429 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.599377 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 05 13:20:49.602633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.602612 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g"] Jun 05 13:20:49.619302 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.619278 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:20:49.719386 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.719350 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c934e323-c86d-43e0-9e4f-87fae783f31a-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.719568 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.719418 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.719568 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.719449 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.719568 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.719523 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.719568 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.719560 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr4w2\" (UniqueName: \"kubernetes.io/projected/c934e323-c86d-43e0-9e4f-87fae783f31a-kube-api-access-gr4w2\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.719744 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.719583 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.820778 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.820740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.820778 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.820778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821012 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.820837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821012 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.820876 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr4w2\" (UniqueName: \"kubernetes.io/projected/c934e323-c86d-43e0-9e4f-87fae783f31a-kube-api-access-gr4w2\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821012 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.820909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821012 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.820945 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c934e323-c86d-43e0-9e4f-87fae783f31a-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821322 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.821294 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.821351 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.821465 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.821391 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.823070 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.823050 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c934e323-c86d-43e0-9e4f-87fae783f31a-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.823340 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.823323 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c934e323-c86d-43e0-9e4f-87fae783f31a-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.830014 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.829992 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr4w2\" (UniqueName: \"kubernetes.io/projected/c934e323-c86d-43e0-9e4f-87fae783f31a-kube-api-access-gr4w2\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ztn9g\" (UID: \"c934e323-c86d-43e0-9e4f-87fae783f31a\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:49.907349 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:49.907229 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:50.042342 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:50.042308 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g"] Jun 05 13:20:50.046121 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:20:50.046087 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc934e323_c86d_43e0_9e4f_87fae783f31a.slice/crio-66c34278e7df139f569a264af29bedfc6a9be015797a43cb39db474f0a4e004c WatchSource:0}: Error finding container 66c34278e7df139f569a264af29bedfc6a9be015797a43cb39db474f0a4e004c: Status 404 returned error can't find the container with id 66c34278e7df139f569a264af29bedfc6a9be015797a43cb39db474f0a4e004c Jun 05 13:20:50.601229 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:50.601188 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" event={"ID":"c934e323-c86d-43e0-9e4f-87fae783f31a","Type":"ContainerStarted","Data":"66c34278e7df139f569a264af29bedfc6a9be015797a43cb39db474f0a4e004c"} Jun 05 13:20:53.619063 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:53.619025 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" event={"ID":"c934e323-c86d-43e0-9e4f-87fae783f31a","Type":"ContainerStarted","Data":"88d611a5ab0791db29e625596c25f97c00b6071b8e084f15ac337e7c2a0b4390"} Jun 05 13:20:55.629519 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:55.629484 2570 generic.go:356] "Generic (PLEG): container finished" podID="c934e323-c86d-43e0-9e4f-87fae783f31a" containerID="88d611a5ab0791db29e625596c25f97c00b6071b8e084f15ac337e7c2a0b4390" exitCode=0 Jun 05 13:20:55.630151 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:55.629535 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" event={"ID":"c934e323-c86d-43e0-9e4f-87fae783f31a","Type":"ContainerDied","Data":"88d611a5ab0791db29e625596c25f97c00b6071b8e084f15ac337e7c2a0b4390"} Jun 05 13:20:57.641314 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:57.641279 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" event={"ID":"c934e323-c86d-43e0-9e4f-87fae783f31a","Type":"ContainerStarted","Data":"60134da895e8d07943cf631014d4b6c46591c1a8a7703e30e955d7204f7a5940"} Jun 05 13:20:57.662626 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:57.662573 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" podStartSLOduration=2.7473055029999998 podStartE2EDuration="8.662558332s" totalImagesPullingTime="4.510254378s" totalInitContainerRuntime="1.404998451s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:20:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:20:57.659800434 +0000 UTC m=+870.301124236" watchObservedRunningTime="2026-06-05 13:20:57.662558332 +0000 UTC m=+870.303882133" Jun 05 13:20:58.399508 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.399478 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k"] Jun 05 13:20:58.403095 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.403068 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.406578 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.406555 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 05 13:20:58.415566 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.415542 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k"] Jun 05 13:20:58.443069 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.443029 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:20:58.504086 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.504055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.504272 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.504102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.504272 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.504122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.504272 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.504230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.504385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.504323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfk4m\" (UniqueName: \"kubernetes.io/projected/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-kube-api-access-xfk4m\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.504385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.504353 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.605597 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.605547 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.605786 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.605663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.605786 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.605739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfk4m\" (UniqueName: \"kubernetes.io/projected/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-kube-api-access-xfk4m\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.605786 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.605772 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.605965 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.605812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.605965 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.605875 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.606446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.606078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.606446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.606148 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.606446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.606407 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.608565 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.608547 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.609039 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.609016 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.613841 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.613815 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfk4m\" (UniqueName: \"kubernetes.io/projected/e9b7caf1-6f17-411c-9fa1-bba5b681b1f2-kube-api-access-xfk4m\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k\" (UID: \"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.716672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.716584 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:20:58.856288 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:58.856243 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k"] Jun 05 13:20:58.858139 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:20:58.858113 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b7caf1_6f17_411c_9fa1_bba5b681b1f2.slice/crio-c8869267b1fbce7b9afb24523c019e5d655796ed15c412eab0235650c665ca31 WatchSource:0}: Error finding container c8869267b1fbce7b9afb24523c019e5d655796ed15c412eab0235650c665ca31: Status 404 returned error can't find the container with id c8869267b1fbce7b9afb24523c019e5d655796ed15c412eab0235650c665ca31 Jun 05 13:20:59.653899 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:59.653860 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" event={"ID":"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2","Type":"ContainerStarted","Data":"22dacda5614b72783c83e06bb4dd15f129e8cf9731e11692b954a5ff0bfa2227"} Jun 05 13:20:59.654089 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:59.653920 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" event={"ID":"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2","Type":"ContainerStarted","Data":"c8869267b1fbce7b9afb24523c019e5d655796ed15c412eab0235650c665ca31"} Jun 05 13:20:59.908100 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:59.908009 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:59.908100 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:59.908073 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:20:59.920932 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:20:59.920886 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:21:00.660452 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:00.660354 2570 generic.go:356] "Generic (PLEG): container finished" podID="e9b7caf1-6f17-411c-9fa1-bba5b681b1f2" containerID="22dacda5614b72783c83e06bb4dd15f129e8cf9731e11692b954a5ff0bfa2227" exitCode=0 Jun 05 13:21:00.660642 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:00.660447 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" event={"ID":"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2","Type":"ContainerDied","Data":"22dacda5614b72783c83e06bb4dd15f129e8cf9731e11692b954a5ff0bfa2227"} Jun 05 13:21:00.677230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:00.677204 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ztn9g" Jun 05 13:21:01.667084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:01.667047 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" event={"ID":"e9b7caf1-6f17-411c-9fa1-bba5b681b1f2","Type":"ContainerStarted","Data":"d7e99d26140f35cb839b2747c5c57f9c5ef74033f7a48e98cbe7a3a869aafdb1"} Jun 05 13:21:01.688138 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:01.688083 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" podStartSLOduration=2.031184896 podStartE2EDuration="3.688066032s" totalImagesPullingTime="294.843963ms" totalInitContainerRuntime="1.362037173s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:21:01.685980684 +0000 UTC m=+874.327304508" watchObservedRunningTime="2026-06-05 13:21:01.688066032 +0000 UTC m=+874.329389832" Jun 05 13:21:08.716746 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:08.716703 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:21:08.717334 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:08.716844 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:21:08.729564 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:08.729531 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:21:09.718230 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:09.718202 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k" Jun 05 13:21:12.056548 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:12.056512 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:21:12.111449 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:12.111404 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:21:13.402424 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.402353 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k"] Jun 05 13:21:13.409446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.409422 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.412114 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.412093 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 05 13:21:13.420960 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.420928 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k"] Jun 05 13:21:13.435500 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.435467 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:21:13.453032 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.453002 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.453209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.453047 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.453209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.453090 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.453209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.453140 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6419cf9e-a390-40ce-bad2-00af9f47c2c6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.453209 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.453161 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8grkb\" (UniqueName: \"kubernetes.io/projected/6419cf9e-a390-40ce-bad2-00af9f47c2c6-kube-api-access-8grkb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.453547 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.453217 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554322 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554402 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554436 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6419cf9e-a390-40ce-bad2-00af9f47c2c6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554522 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554486 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grkb\" (UniqueName: \"kubernetes.io/projected/6419cf9e-a390-40ce-bad2-00af9f47c2c6-kube-api-access-8grkb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554815 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554785 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554815 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554810 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.554976 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.554855 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.556712 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.556692 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6419cf9e-a390-40ce-bad2-00af9f47c2c6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.556931 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.556902 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6419cf9e-a390-40ce-bad2-00af9f47c2c6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.563820 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.563796 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8grkb\" (UniqueName: \"kubernetes.io/projected/6419cf9e-a390-40ce-bad2-00af9f47c2c6-kube-api-access-8grkb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k\" (UID: \"6419cf9e-a390-40ce-bad2-00af9f47c2c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.751578 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.751546 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:13.896864 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:13.896836 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k"] Jun 05 13:21:13.899003 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:21:13.898974 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6419cf9e_a390_40ce_bad2_00af9f47c2c6.slice/crio-ba89086c6df02f1b4b9669861aa8285b74ab44725aca5a89415f417602bb95be WatchSource:0}: Error finding container ba89086c6df02f1b4b9669861aa8285b74ab44725aca5a89415f417602bb95be: Status 404 returned error can't find the container with id ba89086c6df02f1b4b9669861aa8285b74ab44725aca5a89415f417602bb95be Jun 05 13:21:14.737403 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:14.737368 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" event={"ID":"6419cf9e-a390-40ce-bad2-00af9f47c2c6","Type":"ContainerStarted","Data":"7ec87f37475b5588b0ac3a4665359ed9fff35e41fd4034a31c097ec77d41f2db"} Jun 05 13:21:14.737794 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:14.737420 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" event={"ID":"6419cf9e-a390-40ce-bad2-00af9f47c2c6","Type":"ContainerStarted","Data":"ba89086c6df02f1b4b9669861aa8285b74ab44725aca5a89415f417602bb95be"} Jun 05 13:21:15.743304 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:15.743240 2570 generic.go:356] "Generic (PLEG): container finished" podID="6419cf9e-a390-40ce-bad2-00af9f47c2c6" containerID="7ec87f37475b5588b0ac3a4665359ed9fff35e41fd4034a31c097ec77d41f2db" exitCode=0 Jun 05 13:21:15.743710 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:15.743342 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" event={"ID":"6419cf9e-a390-40ce-bad2-00af9f47c2c6","Type":"ContainerDied","Data":"7ec87f37475b5588b0ac3a4665359ed9fff35e41fd4034a31c097ec77d41f2db"} Jun 05 13:21:16.750908 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:16.750872 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" event={"ID":"6419cf9e-a390-40ce-bad2-00af9f47c2c6","Type":"ContainerStarted","Data":"ef11e734e32f233460b90dd89bffc1dbc18b460aff1cb33d67a2675739f0dc21"} Jun 05 13:21:16.774321 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:16.774264 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" podStartSLOduration=2.094785612 podStartE2EDuration="3.774227088s" totalImagesPullingTime="346.213168ms" totalInitContainerRuntime="1.333228308s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:21:16.772825378 +0000 UTC m=+889.414149179" watchObservedRunningTime="2026-06-05 13:21:16.774227088 +0000 UTC m=+889.415550889" Jun 05 13:21:22.330421 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:22.330323 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:21:23.751874 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:23.751838 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:23.752299 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:23.751903 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:23.764626 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:23.764602 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:23.797446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:23.797419 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k" Jun 05 13:21:27.796597 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:27.796567 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:21:27.799104 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:27.799083 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:21:38.243431 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:21:38.243384 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:22:59.920528 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:22:59.920401 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:22:59.981075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:22:59.981035 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:23:06.467356 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:23:06.467297 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:23:06.571127 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:23:06.571101 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:23:07.642782 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:23:07.642753 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:23:07.738956 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:23:07.738920 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:23:09.842097 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:23:09.842059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:23:09.902816 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:23:09.902788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:24:36.166316 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:36.166228 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:24:39.299151 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:39.299103 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:24:44.237776 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.237737 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-2rn5l"] Jun 05 13:24:44.238233 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.237958 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" podUID="25504245-5107-420f-8485-b58484beb2d2" containerName="manager" containerID="cri-o://53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349" gracePeriod=30 Jun 05 13:24:44.491116 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.491061 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:24:44.572554 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.572519 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4r27\" (UniqueName: \"kubernetes.io/projected/25504245-5107-420f-8485-b58484beb2d2-kube-api-access-v4r27\") pod \"25504245-5107-420f-8485-b58484beb2d2\" (UID: \"25504245-5107-420f-8485-b58484beb2d2\") " Jun 05 13:24:44.574517 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.574492 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25504245-5107-420f-8485-b58484beb2d2-kube-api-access-v4r27" (OuterVolumeSpecName: "kube-api-access-v4r27") pod "25504245-5107-420f-8485-b58484beb2d2" (UID: "25504245-5107-420f-8485-b58484beb2d2"). InnerVolumeSpecName "kube-api-access-v4r27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 05 13:24:44.674019 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.673994 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4r27\" (UniqueName: \"kubernetes.io/projected/25504245-5107-420f-8485-b58484beb2d2-kube-api-access-v4r27\") on node \"ip-10-0-141-104.ec2.internal\" DevicePath \"\"" Jun 05 13:24:44.788915 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.788811 2570 generic.go:356] "Generic (PLEG): container finished" podID="25504245-5107-420f-8485-b58484beb2d2" containerID="53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349" exitCode=0 Jun 05 13:24:44.788915 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.788858 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" Jun 05 13:24:44.789084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.788853 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" event={"ID":"25504245-5107-420f-8485-b58484beb2d2","Type":"ContainerDied","Data":"53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349"} Jun 05 13:24:44.789084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.788979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ccf9c474-2rn5l" event={"ID":"25504245-5107-420f-8485-b58484beb2d2","Type":"ContainerDied","Data":"d653eec2cc18e199f766edd14e347bb99de3372e8ce8b39bd04bbf836967272e"} Jun 05 13:24:44.789084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.789002 2570 scope.go:117] "RemoveContainer" containerID="53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349" Jun 05 13:24:44.798349 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.798332 2570 scope.go:117] "RemoveContainer" containerID="53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349" Jun 05 13:24:44.798587 ip-10-0-141-104 kubenswrapper[2570]: E0605 13:24:44.798568 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349\": container with ID starting with 53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349 not found: ID does not exist" containerID="53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349" Jun 05 13:24:44.798634 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.798595 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349"} err="failed to get container status \"53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349\": rpc error: code = NotFound desc = could not find container \"53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349\": container with ID starting with 53c5fd4808e62a46675abd107a53a5f05f1390ede9b537dca70500b96cdca349 not found: ID does not exist" Jun 05 13:24:44.811389 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.811369 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-2rn5l"] Jun 05 13:24:44.815234 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:44.815213 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-2rn5l"] Jun 05 13:24:45.521013 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.520971 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58ccf9c474-ngsbd"] Jun 05 13:24:45.521599 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.521580 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25504245-5107-420f-8485-b58484beb2d2" containerName="manager" Jun 05 13:24:45.521687 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.521602 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="25504245-5107-420f-8485-b58484beb2d2" containerName="manager" Jun 05 13:24:45.521741 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.521708 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="25504245-5107-420f-8485-b58484beb2d2" containerName="manager" Jun 05 13:24:45.526204 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.526182 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:24:45.528514 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.528495 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 05 13:24:45.528627 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.528556 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-k7nrh\"" Jun 05 13:24:45.532517 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.532495 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-ngsbd"] Jun 05 13:24:45.682783 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.682735 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n4ct\" (UniqueName: \"kubernetes.io/projected/ba73ef31-a80f-4d53-a6ad-93cf4156b511-kube-api-access-8n4ct\") pod \"maas-controller-58ccf9c474-ngsbd\" (UID: \"ba73ef31-a80f-4d53-a6ad-93cf4156b511\") " pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:24:45.783766 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.783682 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25504245-5107-420f-8485-b58484beb2d2" path="/var/lib/kubelet/pods/25504245-5107-420f-8485-b58484beb2d2/volumes" Jun 05 13:24:45.784418 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.784381 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n4ct\" (UniqueName: \"kubernetes.io/projected/ba73ef31-a80f-4d53-a6ad-93cf4156b511-kube-api-access-8n4ct\") pod \"maas-controller-58ccf9c474-ngsbd\" (UID: \"ba73ef31-a80f-4d53-a6ad-93cf4156b511\") " pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:24:45.794043 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.794013 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n4ct\" (UniqueName: \"kubernetes.io/projected/ba73ef31-a80f-4d53-a6ad-93cf4156b511-kube-api-access-8n4ct\") pod \"maas-controller-58ccf9c474-ngsbd\" (UID: \"ba73ef31-a80f-4d53-a6ad-93cf4156b511\") " pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:24:45.838998 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.838960 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:24:45.977005 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.976977 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58ccf9c474-ngsbd"] Jun 05 13:24:45.979102 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:24:45.979078 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba73ef31_a80f_4d53_a6ad_93cf4156b511.slice/crio-62699fcfcae4cf7691fbe30ba67b53f1f7713acddd6d43743f6e825fdb798006 WatchSource:0}: Error finding container 62699fcfcae4cf7691fbe30ba67b53f1f7713acddd6d43743f6e825fdb798006: Status 404 returned error can't find the container with id 62699fcfcae4cf7691fbe30ba67b53f1f7713acddd6d43743f6e825fdb798006 Jun 05 13:24:45.980412 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:45.980393 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 05 13:24:46.802531 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:46.802494 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" event={"ID":"ba73ef31-a80f-4d53-a6ad-93cf4156b511","Type":"ContainerStarted","Data":"cb6e6c8c2f908f0d0508a3f080366d89c8216a6a2285c64af25bbef0ec26b934"} Jun 05 13:24:46.802953 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:46.802547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" event={"ID":"ba73ef31-a80f-4d53-a6ad-93cf4156b511","Type":"ContainerStarted","Data":"62699fcfcae4cf7691fbe30ba67b53f1f7713acddd6d43743f6e825fdb798006"} Jun 05 13:24:46.802953 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:46.802617 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:24:46.820198 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:46.820143 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" podStartSLOduration=1.339840647 podStartE2EDuration="1.820124615s" totalImagesPullingTime="480.283968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-05 13:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:24:46.819627686 +0000 UTC m=+1099.460951490" watchObservedRunningTime="2026-06-05 13:24:46.820124615 +0000 UTC m=+1099.461448415" Jun 05 13:24:57.813045 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:24:57.813017 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58ccf9c474-ngsbd" Jun 05 13:25:25.237819 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:25:25.237762 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:25:25.336995 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:25:25.336960 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:25:33.585712 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:25:33.585678 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:25:39.796359 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:25:39.796322 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:25:39.902107 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:25:39.902072 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:11.857264 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:11.857162 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:11.957196 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:11.957163 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:27.831805 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:27.831777 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:26:27.835556 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:27.835535 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:26:35.661337 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:35.661303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:41.432167 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:41.432133 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:41.494353 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:41.494316 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:41.587347 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:41.587315 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:41.686351 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:41.686237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:41.849951 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:41.849904 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:41.953895 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:41.953820 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:58.254583 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:58.254549 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:58.354750 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:58.354711 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:26:58.466241 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:26:58.466207 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:04.338146 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:04.338110 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:04.437785 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:04.437750 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:12.489305 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:12.489218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:12.592588 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:12.592555 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:12.692897 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:12.692860 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:12.794471 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:12.794388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:29.216652 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:29.216564 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:29.316882 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:29.316840 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:29.430180 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:29.430146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:35.175462 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:35.175430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:27:35.302065 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:27:35.302029 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:28:28.157606 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:28:28.157565 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:28:28.255497 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:28:28.255461 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:28:33.329721 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:28:33.329676 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:04.216916 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:04.216883 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:15.796327 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:15.796292 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:15.897789 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:15.897753 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:38.071911 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:38.071877 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:38.133527 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:38.133493 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:38.698614 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:38.698584 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:38.776655 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:38.776621 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:30:59.931509 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:30:59.931460 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:31:00.031560 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:31:00.031527 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:31:27.869493 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:31:27.869461 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:31:27.874222 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:31:27.874196 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:32:52.742139 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:32:52.742106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:33:53.700892 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:33:53.700856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:33:53.751823 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:33:53.751787 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:36:27.902220 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:36:27.902187 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:36:27.907489 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:36:27.907463 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:38:59.932872 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:38:59.932829 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:39:01.127997 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:39:01.127967 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:39:07.824110 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:39:07.824083 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:39:07.925463 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:39:07.925436 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:39:26.923445 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:39:26.923372 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:39:26.988139 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:39:26.988108 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:41:27.933883 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:41:27.933850 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:41:27.942940 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:41:27.942920 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:41:42.328705 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:41:42.328670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:41:57.968443 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:41:57.968412 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:41:58.068110 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:41:58.068083 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:42:35.877918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:42:35.877837 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:43:33.831295 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:43:33.831264 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:43:33.931635 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:43:33.931603 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:43:52.902281 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:43:52.902191 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:43:53.001933 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:43:53.001898 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:44:33.836833 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:44:33.836786 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:44:33.933346 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:44:33.933312 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:44:52.913644 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:44:52.913612 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:44:52.980125 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:44:52.980076 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:17.520407 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:17.520321 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:26.899599 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:26.899564 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:27.002749 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:27.002720 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:27.099635 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:27.099604 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:33.852309 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:33.852266 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:33.919385 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:33.919357 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:41.918502 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:41.918468 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:41.986910 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:41.986874 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:51.306440 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:51.306402 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:52.940293 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:52.940237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:53.019492 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:53.019460 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:56.030021 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:56.029989 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:56.130876 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:56.130843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:56.327632 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:56.327552 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:45:56.431993 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:45:56.431963 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:04.392552 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:04.392519 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:06.041147 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:06.041118 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:06.144317 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:06.144276 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:06.247722 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:06.247647 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:09.601305 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:09.601269 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:12.316890 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:12.316861 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:12.523177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:12.523149 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:12.679163 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:12.679078 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:13.835065 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:13.835033 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:19.710514 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:19.710479 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:19.814177 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:19.814145 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:19.992348 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:19.992321 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:20.139795 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:20.139763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:20.244082 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:20.243985 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:23.333541 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:23.333503 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:26.193602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:26.193568 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:27.385158 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:27.385111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:27.465017 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:27.464985 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:27.967587 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:27.967478 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:46:27.975907 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:27.975888 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:46:29.822519 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:29.822477 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:33.855891 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:33.855859 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:33.960216 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:33.960179 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:45.040418 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:45.040391 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-56fcb9bb9-5mmfz_6e88bf51-725e-458e-91e1-9d9d06b83cfc/maas-api/0.log" Jun 05 13:46:45.369139 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:45.369052 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58ccf9c474-ngsbd_ba73ef31-a80f-4d53-a6ad-93cf4156b511/manager/0.log" Jun 05 13:46:45.951280 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:45.951219 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b9f446d7-t69wv_855c3beb-cda0-4b33-a930-c53c4c2b4310/copy-manifests/0.log" Jun 05 13:46:45.959001 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:45.958979 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b9f446d7-t69wv_855c3beb-cda0-4b33-a930-c53c4c2b4310/manager/0.log" Jun 05 13:46:46.070091 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:46.070050 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6mdxb_66bdc07f-e31b-40c9-972a-77919bd3187a/postgres/0.log" Jun 05 13:46:48.711378 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:48.711296 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f65db7bc7-8jh2v_9054225c-7b02-48a0-983e-fb671bbc58bd/kube-auth-proxy/0.log" Jun 05 13:46:48.934215 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:48.934182 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/1.log" Jun 05 13:46:49.047403 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:49.047373 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-nm7nl_60724d9d-0330-45d1-9d8d-e5e756f29a60/payload-pre-processing/0.log" Jun 05 13:46:49.156899 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:49.156854 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-gc4zb_33bbb2ca-3289-4f75-b469-5776d125a755/payload-processing/0.log" Jun 05 13:46:49.769688 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:49.769661 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-ztn9g_c934e323-c86d-43e0-9e4f-87fae783f31a/storage-initializer/0.log" Jun 05 13:46:49.781691 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:49.781666 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-ztn9g_c934e323-c86d-43e0-9e4f-87fae783f31a/main/0.log" Jun 05 13:46:49.899418 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:49.899385 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k_e9b7caf1-6f17-411c-9fa1-bba5b681b1f2/main/0.log" Jun 05 13:46:49.906852 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:49.906829 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-t6n2k_e9b7caf1-6f17-411c-9fa1-bba5b681b1f2/storage-initializer/0.log" Jun 05 13:46:50.021634 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:50.021541 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k_6419cf9e-a390-40ce-bad2-00af9f47c2c6/storage-initializer/0.log" Jun 05 13:46:50.031212 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:50.031191 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cswf7k_6419cf9e-a390-40ce-bad2-00af9f47c2c6/main/0.log" Jun 05 13:46:54.910084 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:54.910014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:56.108674 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:56.108641 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:46:58.337087 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:58.337055 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7q58h_840dd6b5-5015-4128-be30-aa29c74fa1e8/global-pull-secret-syncer/0.log" Jun 05 13:46:58.496411 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:58.496385 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" Jun 05 13:46:58.644188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:46:58.644115 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-104.ec2.internal_24f75fd8523075e78d67ffee29b41dde/haproxy/0.log" Jun 05 13:47:06.684802 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:06.684700 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-q2vgx_295dbce6-3dc9-42bc-b917-8c6f21a339eb/cluster-monitoring-operator/0.log" Jun 05 13:47:06.730394 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:06.730365 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-28lpr_73530901-1d93-43e1-b18e-99ee84a5e052/kube-state-metrics/0.log" Jun 05 13:47:06.765675 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:06.765644 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-28lpr_73530901-1d93-43e1-b18e-99ee84a5e052/kube-rbac-proxy-main/0.log" Jun 05 13:47:06.799144 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:06.799108 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-28lpr_73530901-1d93-43e1-b18e-99ee84a5e052/kube-rbac-proxy-self/0.log" Jun 05 13:47:06.843381 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:06.843342 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65cd568d69-nzcpg_4e7b7b3a-c15b-40e8-99b5-ed03042928e4/metrics-server/0.log" Jun 05 13:47:06.898929 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:06.898886 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-nkb9s_8fe1fbde-e3fb-4c4b-b359-387de7f15ac9/monitoring-plugin/0.log" Jun 05 13:47:07.047573 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.047540 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grznq_03399703-6846-44f2-856c-614e1df5e51c/node-exporter/0.log" Jun 05 13:47:07.078151 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.078130 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grznq_03399703-6846-44f2-856c-614e1df5e51c/kube-rbac-proxy/0.log" Jun 05 13:47:07.112665 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.112642 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-grznq_03399703-6846-44f2-856c-614e1df5e51c/init-textfile/0.log" Jun 05 13:47:07.368162 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.368067 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/prometheus/0.log" Jun 05 13:47:07.412717 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.412672 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/config-reloader/0.log" Jun 05 13:47:07.447322 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.447293 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/thanos-sidecar/0.log" Jun 05 13:47:07.478672 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.478647 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/kube-rbac-proxy-web/0.log" Jun 05 13:47:07.505166 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.505142 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/kube-rbac-proxy/0.log" Jun 05 13:47:07.531458 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.531436 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/kube-rbac-proxy-thanos/0.log" Jun 05 13:47:07.558643 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.558620 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8e7b14e-9514-44cc-acd1-05e4100cac67/init-config-reloader/0.log" Jun 05 13:47:07.592708 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.592689 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-ggbb7_0d2361f9-9279-41de-b218-d4d64d23699c/prometheus-operator/0.log" Jun 05 13:47:07.638945 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.638875 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-ggbb7_0d2361f9-9279-41de-b218-d4d64d23699c/kube-rbac-proxy/0.log" Jun 05 13:47:07.675402 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.675382 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-rcgd9_4239b006-96bc-4178-a4f6-a9548fd18b91/prometheus-operator-admission-webhook/0.log" Jun 05 13:47:07.726395 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.726372 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c6897df99-7x5ng_c2170061-4cfd-471e-8b0e-89282819502f/telemeter-client/0.log" Jun 05 13:47:07.784197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.784170 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c6897df99-7x5ng_c2170061-4cfd-471e-8b0e-89282819502f/reload/0.log" Jun 05 13:47:07.830377 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:07.830334 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c6897df99-7x5ng_c2170061-4cfd-471e-8b0e-89282819502f/kube-rbac-proxy/0.log" Jun 05 13:47:09.452476 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.452447 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-lpd2g_611f6ea8-f9d0-47dc-8c94-6f1d1653e6ac/networking-console-plugin/0.log" Jun 05 13:47:09.500189 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.500160 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl"] Jun 05 13:47:09.503446 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.503427 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.507093 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.507076 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nb2f\"/\"kube-root-ca.crt\"" Jun 05 13:47:09.507608 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.507589 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nb2f\"/\"openshift-service-ca.crt\"" Jun 05 13:47:09.508182 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.508163 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4nb2f\"/\"default-dockercfg-bfjnb\"" Jun 05 13:47:09.531017 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.530993 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl"] Jun 05 13:47:09.599904 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.599875 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-lib-modules\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.600075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.599982 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-sys\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.600075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.600019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-proc\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.600075 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.600060 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-podres\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.600184 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.600091 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6bn\" (UniqueName: \"kubernetes.io/projected/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-kube-api-access-9m6bn\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700725 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-sys\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700735 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-proc\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-podres\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6bn\" (UniqueName: \"kubernetes.io/projected/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-kube-api-access-9m6bn\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700801 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-lib-modules\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700820 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-proc\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700821 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-sys\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.700918 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700880 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-podres\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.701228 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.700944 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-lib-modules\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.710162 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.710106 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6bn\" (UniqueName: \"kubernetes.io/projected/3a3e14cd-b2c7-4100-b698-6a68b052b0cf-kube-api-access-9m6bn\") pod \"perf-node-gather-daemonset-kffwl\" (UID: \"3a3e14cd-b2c7-4100-b698-6a68b052b0cf\") " pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.812755 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.812723 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:09.955605 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:09.955583 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl"] Jun 05 13:47:09.957268 ip-10-0-141-104 kubenswrapper[2570]: W0605 13:47:09.957220 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3a3e14cd_b2c7_4100_b698_6a68b052b0cf.slice/crio-a54cd59378b83d52f72981a047377d935863e9a3c4633c0beaff1e06a9519cbe WatchSource:0}: Error finding container a54cd59378b83d52f72981a047377d935863e9a3c4633c0beaff1e06a9519cbe: Status 404 returned error can't find the container with id a54cd59378b83d52f72981a047377d935863e9a3c4633c0beaff1e06a9519cbe Jun 05 13:47:10.235633 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:10.235608 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/2.log" Jun 05 13:47:10.239214 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:10.239193 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jjn5c_75838c9e-e3ab-4c39-848c-528d0dcf1fac/console-operator/3.log" Jun 05 13:47:10.595479 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:10.595387 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" event={"ID":"3a3e14cd-b2c7-4100-b698-6a68b052b0cf","Type":"ContainerStarted","Data":"66ff306f4d871aa4decc220907823f68cc9117751f5607bf62e5741749091dc2"} Jun 05 13:47:10.595479 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:10.595438 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" event={"ID":"3a3e14cd-b2c7-4100-b698-6a68b052b0cf","Type":"ContainerStarted","Data":"a54cd59378b83d52f72981a047377d935863e9a3c4633c0beaff1e06a9519cbe"} Jun 05 13:47:10.595978 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:10.595494 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:10.620542 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:10.620481 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" podStartSLOduration=1.620460968 podStartE2EDuration="1.620460968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-05 13:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-05 13:47:10.619161911 +0000 UTC m=+2443.260485703" watchObservedRunningTime="2026-06-05 13:47:10.620460968 +0000 UTC m=+2443.261784769" Jun 05 13:47:11.629188 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:11.629154 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-42xdr_92e0b03a-57f1-4269-bafb-d857d716d238/volume-data-source-validator/0.log" Jun 05 13:47:12.871018 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:12.870989 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t27mz_a6d4bb4b-66ac-4457-99e4-1acafbf1d98e/dns/0.log" Jun 05 13:47:12.912523 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:12.912494 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t27mz_a6d4bb4b-66ac-4457-99e4-1acafbf1d98e/kube-rbac-proxy/0.log" Jun 05 13:47:13.068097 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:13.068059 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2d6ll_e3e1da7e-c5f3-4a04-95c8-230051ddcafe/dns-node-resolver/0.log" Jun 05 13:47:13.808314 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:13.808288 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zg2xk_3a2c2470-f752-457d-b759-fad45632ee5f/node-ca/0.log" Jun 05 13:47:14.952786 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:14.952761 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f65db7bc7-8jh2v_9054225c-7b02-48a0-983e-fb671bbc58bd/kube-auth-proxy/0.log" Jun 05 13:47:15.012866 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:15.012837 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/0.log" Jun 05 13:47:15.041524 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:15.041495 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/1.log" Jun 05 13:47:15.091763 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:15.091726 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-nm7nl_60724d9d-0330-45d1-9d8d-e5e756f29a60/payload-pre-processing/0.log" Jun 05 13:47:15.159799 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:15.159771 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-gc4zb_33bbb2ca-3289-4f75-b469-5776d125a755/payload-processing/0.log" Jun 05 13:47:15.887625 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:15.887598 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d2nvt_f5dcc392-3021-4ab6-8b13-a85ec0cfedf4/serve-healthcheck-canary/0.log" Jun 05 13:47:16.524761 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:16.524717 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-tsj2d_ee227f79-486b-42c0-9500-cbdb8540900c/insights-operator/0.log" Jun 05 13:47:16.526358 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:16.526341 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-tsj2d_ee227f79-486b-42c0-9500-cbdb8540900c/insights-operator/1.log" Jun 05 13:47:16.610118 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:16.610092 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4nb2f/perf-node-gather-daemonset-kffwl" Jun 05 13:47:16.824357 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:16.824279 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nl9z6_32d63087-1325-4157-82dd-8e570421d61e/kube-rbac-proxy/0.log" Jun 05 13:47:16.852643 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:16.852619 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nl9z6_32d63087-1325-4157-82dd-8e570421d61e/exporter/0.log" Jun 05 13:47:16.887613 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:16.887583 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nl9z6_32d63087-1325-4157-82dd-8e570421d61e/extractor/0.log" Jun 05 13:47:18.914767 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:18.914736 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:47:19.899197 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:19.899166 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-56fcb9bb9-5mmfz_6e88bf51-725e-458e-91e1-9d9d06b83cfc/maas-api/0.log" Jun 05 13:47:20.054363 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:20.054331 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58ccf9c474-ngsbd_ba73ef31-a80f-4d53-a6ad-93cf4156b511/manager/0.log" Jun 05 13:47:20.386469 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:20.386441 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b9f446d7-t69wv_855c3beb-cda0-4b33-a930-c53c4c2b4310/manager/0.log" Jun 05 13:47:20.411486 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:20.411455 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b9f446d7-t69wv_855c3beb-cda0-4b33-a930-c53c4c2b4310/copy-manifests/0.log" Jun 05 13:47:20.436658 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:20.436627 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6mdxb_66bdc07f-e31b-40c9-972a-77919bd3187a/postgres/0.log" Jun 05 13:47:21.923425 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:21.923394 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5r9xd_d39febf5-14be-40b7-9861-bea08e0c2405/konnectivity-agent/0.log" limit=1024 Jun 05 13:47:22.166308 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:22.166283 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b5dbd5dd-plrql_6d41eada-d384-49fb-b319-d8fbc1b8726a/manager/0.log" Jun 05 13:47:23.583620 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:23.583590 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f65db7bc7-8jh2v_9054225c-7b02-48a0-983e-fb671bbc58bd/kube-auth-proxy/0.log" Jun 05 13:47:23.643206 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:23.643175 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/0.log" Jun 05 13:47:23.674846 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:23.674817 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/1.log" Jun 05 13:47:23.708414 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:23.708386 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-nm7nl_60724d9d-0330-45d1-9d8d-e5e756f29a60/payload-pre-processing/0.log" Jun 05 13:47:23.756775 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:23.756749 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-gc4zb_33bbb2ca-3289-4f75-b469-5776d125a755/payload-processing/0.log" Jun 05 13:47:24.987990 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:24.987965 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f65db7bc7-8jh2v_9054225c-7b02-48a0-983e-fb671bbc58bd/kube-auth-proxy/0.log" Jun 05 13:47:25.036602 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:25.036571 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/0.log" Jun 05 13:47:25.065226 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:25.065197 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/1.log" Jun 05 13:47:25.095118 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:25.095092 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-nm7nl_60724d9d-0330-45d1-9d8d-e5e756f29a60/payload-pre-processing/0.log" Jun 05 13:47:25.145504 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:25.145475 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-gc4zb_33bbb2ca-3289-4f75-b469-5776d125a755/payload-processing/0.log" Jun 05 13:47:27.054260 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:27.054226 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f65db7bc7-8jh2v_9054225c-7b02-48a0-983e-fb671bbc58bd/kube-auth-proxy/0.log" Jun 05 13:47:27.102877 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:27.102838 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/0.log" Jun 05 13:47:27.130758 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:27.130726 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zxjrd_8163f9c8-dcf3-4d8e-807d-0b40b2b87048/istio-proxy/1.log" Jun 05 13:47:27.162532 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:27.162504 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-nm7nl_60724d9d-0330-45d1-9d8d-e5e756f29a60/payload-pre-processing/0.log" Jun 05 13:47:27.209110 ip-10-0-141-104 kubenswrapper[2570]: I0605 13:47:27.209083 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-gc4zb_33bbb2ca-3289-4f75-b469-5776d125a755/payload-processing/0.log"