Jun 09 11:52:20.764230 ip-10-0-139-166 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 09 11:52:20.764245 ip-10-0-139-166 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 09 11:52:20.764254 ip-10-0-139-166 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 09 11:52:20.764480 ip-10-0-139-166 systemd[1]: Failed to start Kubernetes Kubelet. Jun 09 11:52:30.954047 ip-10-0-139-166 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 09 11:52:30.954061 ip-10-0-139-166 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5c07694cccb045f8887b0e6503115135 -- Jun 09 11:54:50.591028 ip-10-0-139-166 systemd[1]: Starting Kubernetes Kubelet... Jun 09 11:54:51.035043 ip-10-0-139-166 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 11:54:51.035043 ip-10-0-139-166 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 11:54:51.035043 ip-10-0-139-166 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 11:54:51.035043 ip-10-0-139-166 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 11:54:51.035043 ip-10-0-139-166 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 11:54:51.038015 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.037925 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 11:54:51.044811 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044795 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:54:51.044811 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044811 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044816 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044819 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044822 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044825 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044828 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044832 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044835 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044837 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044840 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044843 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044846 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044848 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044851 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044855 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044857 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044860 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044870 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044873 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044876 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:54:51.044878 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044879 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044882 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044885 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044888 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044890 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044893 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044896 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044899 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044901 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044904 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044906 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044909 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044911 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044914 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044917 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044919 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044922 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044924 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044927 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044929 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:54:51.045349 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044932 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044934 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044936 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044939 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044943 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044946 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044949 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044952 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044956 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044959 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044962 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044965 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044968 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044971 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044975 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044978 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044981 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044984 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044987 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:54:51.045835 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044989 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044992 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044995 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.044998 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045000 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045003 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045006 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045008 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045010 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045013 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045016 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045018 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045020 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045023 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045025 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045028 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045030 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045033 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045036 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045038 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:54:51.046300 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045041 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045043 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045046 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045048 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045052 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045054 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045057 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045060 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045064 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045068 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045072 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045075 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045079 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045083 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045086 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045089 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045092 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045095 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:54:51.046787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.045098 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046673 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046682 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046688 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046691 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046694 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046697 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046700 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046703 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046706 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046710 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046714 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046718 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046721 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046724 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046726 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046729 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046748 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046751 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:54:51.047250 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046755 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046757 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046760 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046763 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046766 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046770 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046774 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046777 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046779 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046782 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046785 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046787 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046790 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046793 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046796 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046798 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046801 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046804 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046807 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:54:51.047787 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046809 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046812 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046814 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046817 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046819 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046822 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046824 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046827 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046829 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046832 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046835 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046837 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046840 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046842 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046845 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046849 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046852 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046854 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046858 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046860 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:54:51.048254 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046863 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046865 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046868 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046871 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046873 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046876 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046878 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046881 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046884 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046886 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046889 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046891 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046894 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046896 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046899 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046901 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046904 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046906 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046909 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:54:51.048757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046912 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046914 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046916 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046919 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046921 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046924 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046926 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046928 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046932 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046935 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046938 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046940 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046943 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046946 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046948 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046951 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046953 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046955 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046958 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046960 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046964 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:54:51.049226 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046966 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.046969 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047329 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047333 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047336 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047339 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047344 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047347 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047349 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047352 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047355 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047357 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047360 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047362 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047366 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047368 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047371 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047374 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047376 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047379 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:54:51.049757 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047382 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047384 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047387 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047390 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047392 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047395 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047398 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047401 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047403 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047405 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047408 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047411 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047415 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047418 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047421 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047423 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047426 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047428 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047432 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047435 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:54:51.051016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047438 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047440 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047443 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047445 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047448 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047450 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047453 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047456 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047458 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047461 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047463 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047466 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047472 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047475 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047478 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047480 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047482 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047485 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047487 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047490 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:54:51.051510 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047492 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047495 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047498 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047500 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047503 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047505 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047508 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047510 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047513 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047516 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047520 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047522 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047525 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047528 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047530 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047532 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047535 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047538 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047541 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047543 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:54:51.052016 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047546 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047548 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047551 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047554 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047558 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047562 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047565 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047567 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047570 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047573 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047575 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047578 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047580 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047583 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047586 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047588 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047591 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047594 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047598 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:54:51.052500 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047601 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.047604 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047678 2550 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047686 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047693 2550 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047697 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047701 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047705 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047709 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047713 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047718 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047721 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047724 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047727 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047730 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047746 2550 flags.go:64] FLAG: --cgroup-root="" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047749 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047752 2550 flags.go:64] FLAG: --client-ca-file="" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047755 2550 flags.go:64] FLAG: --cloud-provider="external" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047759 2550 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047763 2550 flags.go:64] FLAG: --cluster-domain="" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047766 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047769 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047772 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 11:54:51.052999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047776 2550 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047780 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047783 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047786 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047789 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047792 2550 flags.go:64] FLAG: --contention-profiling="false" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047796 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047799 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047802 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047804 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047808 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047812 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047815 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047818 2550 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047821 2550 flags.go:64] FLAG: --enable-server="true" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047823 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047828 2550 flags.go:64] FLAG: --event-burst="100" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047831 2550 flags.go:64] FLAG: --event-qps="50" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047835 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047838 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047841 2550 flags.go:64] FLAG: --eviction-hard="" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047845 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047848 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047851 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047854 2550 flags.go:64] FLAG: --eviction-soft="" Jun 09 11:54:51.053579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047857 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047859 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047862 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047865 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047868 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047871 2550 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047874 2550 flags.go:64] FLAG: --feature-gates="" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047877 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047880 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047883 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047886 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047889 2550 flags.go:64] FLAG: --healthz-port="10248" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047891 2550 flags.go:64] FLAG: --help="false" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047894 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047897 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047900 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047903 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047906 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047910 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047913 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047915 2550 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047918 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047921 2550 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 11:54:51.054186 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047924 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047927 2550 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047931 2550 flags.go:64] FLAG: --kube-reserved="" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047934 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047936 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047939 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047942 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047945 2550 flags.go:64] FLAG: --lock-file="" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047948 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047951 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047954 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047959 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047962 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047965 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047968 2550 flags.go:64] FLAG: --logging-format="text" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047971 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047974 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047977 2550 flags.go:64] FLAG: --manifest-url="" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047980 2550 flags.go:64] FLAG: --manifest-url-header="" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047984 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047987 2550 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047991 2550 flags.go:64] FLAG: --max-pods="110" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047994 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.047997 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048000 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 11:54:51.054831 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048002 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048005 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048009 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048012 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048018 2550 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048021 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048024 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048027 2550 flags.go:64] FLAG: --pod-cidr="" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048030 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048037 2550 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048040 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048043 2550 flags.go:64] FLAG: --pods-per-core="0" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048046 2550 flags.go:64] FLAG: --port="10250" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048049 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048052 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09248ff7da2db751e" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048054 2550 flags.go:64] FLAG: --qos-reserved="" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048058 2550 flags.go:64] FLAG: --read-only-port="10255" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048060 2550 flags.go:64] FLAG: --register-node="true" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048063 2550 flags.go:64] FLAG: --register-with-taints="" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048067 2550 flags.go:64] FLAG: --registry-burst="10" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048069 2550 flags.go:64] FLAG: --registry-qps="5" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048073 2550 flags.go:64] FLAG: --reserved-cpus="" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048076 2550 flags.go:64] FLAG: --reserved-memory="" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048079 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048082 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 11:54:51.055492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048085 2550 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048088 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048090 2550 flags.go:64] FLAG: --runonce="false" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048093 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048096 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048099 2550 flags.go:64] FLAG: --seccomp-default="false" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048101 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048104 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048107 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048110 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048113 2550 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048116 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048119 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048121 2550 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048124 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048127 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048131 2550 flags.go:64] FLAG: --system-cgroups="" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048134 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048139 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048142 2550 flags.go:64] FLAG: --tls-cert-file="" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048145 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048149 2550 flags.go:64] FLAG: --tls-min-version="" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048152 2550 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048154 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048157 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 11:54:51.056163 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048160 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048163 2550 flags.go:64] FLAG: --v="2" Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048166 2550 flags.go:64] FLAG: --version="false" Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048170 2550 flags.go:64] FLAG: --vmodule="" Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048174 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.048177 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048311 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048316 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048320 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048324 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048327 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048330 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048333 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048336 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048338 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048341 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048343 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048347 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048349 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048352 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:54:51.056798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048356 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048359 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048363 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048366 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048369 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048371 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048374 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048376 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048379 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048381 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048384 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048387 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048389 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048392 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048395 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048397 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048399 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048402 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048405 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048407 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:54:51.057274 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048410 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048412 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048415 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048418 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048420 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048423 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048426 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048428 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048431 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048435 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048438 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048441 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048443 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048446 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048449 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048451 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048454 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048456 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048459 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:54:51.057798 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048461 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048464 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048466 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048469 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048471 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048474 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048476 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048479 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048481 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048484 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048486 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048489 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048492 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048495 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048497 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048500 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048502 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048505 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048507 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048510 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:54:51.058478 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048513 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048515 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048519 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048522 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048524 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048527 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048530 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048532 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048535 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048537 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048540 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048543 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048545 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048548 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048551 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048553 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048556 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048558 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048561 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:54:51.059202 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048563 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048565 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048568 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048570 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048573 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048576 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.048579 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.049518 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.056616 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.056728 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.057670 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.057678 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057748 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057756 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:54:51.059681 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057759 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057763 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057766 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057769 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057772 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057774 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057777 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057779 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057782 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057785 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057788 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057790 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057793 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057795 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057798 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057800 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057803 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057806 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057809 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:54:51.060049 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057811 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057815 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057818 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057820 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057824 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057826 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057830 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057832 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057835 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057838 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057840 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057843 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057845 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057848 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057851 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057853 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057856 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057858 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057860 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057863 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:54:51.060545 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057866 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057868 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057871 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057874 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057876 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057879 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057881 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057884 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057886 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057889 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057891 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057894 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057896 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057899 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057901 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057904 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057906 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.057944 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058039 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058044 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:54:51.061060 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058047 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058051 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058057 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058064 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058070 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058075 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058080 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058084 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058089 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058093 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058098 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058111 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058118 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058126 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058133 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058140 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058148 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058156 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058164 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058172 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:54:51.061539 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058179 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058186 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058194 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058207 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058214 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058221 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058228 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058236 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058243 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058254 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058279 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058287 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058296 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058304 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058312 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058325 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058332 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:54:51.062047 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058340 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.058350 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058757 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058766 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058771 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058775 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058779 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058790 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058794 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058798 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058803 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058807 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058811 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058815 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058820 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:54:51.062490 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058824 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058829 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058833 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058838 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058842 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058853 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058857 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058861 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058865 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058869 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058874 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058880 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058885 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058888 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058893 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058896 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058905 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058914 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058918 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058923 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:54:51.062882 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058927 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058931 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058936 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058940 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058944 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058948 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058953 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058960 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058987 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.058992 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059001 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059006 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059010 2550 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059014 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059019 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059023 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059027 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059031 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059035 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:54:51.063367 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059040 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059064 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059081 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059087 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059111 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059127 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059247 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059254 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059257 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059260 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059263 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059265 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059268 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059271 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059274 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059276 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059279 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059282 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059285 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059287 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:54:51.063844 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059290 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059293 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059296 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059299 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059301 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059304 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059307 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059310 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059315 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059318 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059320 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059323 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059326 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059329 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059331 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059334 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059339 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059343 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059346 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:54:51.064317 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059350 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059353 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059355 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059358 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059361 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059364 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059367 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:51.059369 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.059375 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.059496 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.061873 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.063554 2550 server.go:1019] "Starting client certificate rotation" Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.063647 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 11:54:51.064876 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.063683 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 11:54:51.066384 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.066371 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 11:54:51.069346 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.069328 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 11:54:51.082523 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.082506 2550 log.go:25] "Validated CRI v1 runtime API" Jun 09 11:54:51.085509 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.085493 2550 log.go:25] "Validated CRI v1 image API" Jun 09 11:54:51.087212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.087197 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 11:54:51.089746 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.089706 2550 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7d18626a-48ba-4a94-90ad-ec15525cf793:/dev/nvme0n1p4 a7fdfe84-09ea-4037-ae11-487c56806fb3:/dev/nvme0n1p3] Jun 09 11:54:51.089819 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.089731 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 11:54:51.093224 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.093208 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 11:54:51.095893 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.095786 2550 manager.go:216] Machine: {Timestamp:2026-06-09 11:54:51.094185843 +0000 UTC m=+0.376003064 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103143 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d965a4bd467b9038707ee1e0a08d6 SystemUUID:ec2d965a-4bd4-67b9-0387-07ee1e0a08d6 BootID:5c07694c-ccb0-45f8-887b-0e6503115135 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b6:9b:62:d1:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b6:9b:62:d1:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:6a:de:2c:e6:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 11:54:51.095893 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.095888 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 11:54:51.096002 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.095993 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 11:54:51.098598 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.098574 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 11:54:51.098756 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.098600 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-166.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 11:54:51.100018 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.100008 2550 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 11:54:51.100050 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.100020 2550 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 11:54:51.100050 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.100028 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 11:54:51.100050 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.100043 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 11:54:51.101612 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.101603 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 11:54:51.105703 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105692 2550 state_mem.go:36] "Initialized new in-memory state store" Jun 09 11:54:51.105827 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105818 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 11:54:51.105856 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105850 2550 kubelet.go:486] "Attempting to sync node with API server" Jun 09 11:54:51.105883 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105862 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 11:54:51.105883 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105879 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 11:54:51.105944 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105892 2550 kubelet.go:398] "Adding apiserver pod source" Jun 09 11:54:51.105944 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.105904 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 11:54:51.106301 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.106287 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 11:54:51.106335 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.106316 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 11:54:51.109971 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.109947 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 11:54:51.111791 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.111777 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 11:54:51.111853 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.111807 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 11:54:51.113206 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113194 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113210 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113216 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113221 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113230 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113235 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113243 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 11:54:51.113246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113248 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 11:54:51.113416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113255 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 11:54:51.113416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113260 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 11:54:51.113416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113269 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 11:54:51.113416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.113274 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 11:54:51.114176 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.114166 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 11:54:51.114176 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.114176 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 11:54:51.117101 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.117087 2550 server.go:1268] "Started kubelet" Jun 09 11:54:51.117313 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.117181 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 11:54:51.117313 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.117203 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 11:54:51.117313 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.117251 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 11:54:51.117995 ip-10-0-139-166 systemd[1]: Started Kubernetes Kubelet. Jun 09 11:54:51.119455 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.119432 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 11:54:51.119819 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.119801 2550 server.go:310] "Adding debug handlers to kubelet server" Jun 09 11:54:51.120310 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.120284 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-166.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 11:54:51.120446 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.120416 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 11:54:51.120552 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.120474 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-166.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 11:54:51.125519 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.125501 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 11:54:51.126392 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.126374 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 11:54:51.126392 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.126383 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 11:54:51.126523 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.126438 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.127040 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.127023 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 11:54:51.127040 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.127026 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 11:54:51.127180 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.127049 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 11:54:51.127180 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.127141 2550 reconstruct.go:97] "Volume reconstruction finished" Jun 09 11:54:51.127180 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.127149 2550 reconciler.go:29] "Reconciler: start to sync state" Jun 09 11:54:51.127321 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.126164 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-166.ec2.internal.18b7679a8bd6ead8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-166.ec2.internal,UID:ip-10-0-139-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-166.ec2.internal,},FirstTimestamp:2026-06-09 11:54:51.117062872 +0000 UTC m=+0.398880085,LastTimestamp:2026-06-09 11:54:51.117062872 +0000 UTC m=+0.398880085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-166.ec2.internal,}" Jun 09 11:54:51.128049 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128033 2550 factory.go:55] Registering systemd factory Jun 09 11:54:51.128049 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128050 2550 factory.go:223] Registration of the systemd container factory successfully Jun 09 11:54:51.128248 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128234 2550 factory.go:153] Registering CRI-O factory Jun 09 11:54:51.128311 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128250 2550 factory.go:223] Registration of the crio container factory successfully Jun 09 11:54:51.128311 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128292 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 11:54:51.128394 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128314 2550 factory.go:103] Registering Raw factory Jun 09 11:54:51.128394 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128331 2550 manager.go:1196] Started watching for new ooms in manager Jun 09 11:54:51.128782 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.128768 2550 manager.go:349] Starting recovery of all containers Jun 09 11:54:51.135495 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.135455 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 11:54:51.137075 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.137043 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-166.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 09 11:54:51.137165 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.137143 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 09 11:54:51.142900 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.142882 2550 manager.go:354] Recovery completed Jun 09 11:54:51.146783 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.146770 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.148554 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.148532 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.148627 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.148573 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.148627 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.148590 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.149127 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.149110 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 11:54:51.149127 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.149127 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 11:54:51.149252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.149144 2550 state_mem.go:36] "Initialized new in-memory state store" Jun 09 11:54:51.149252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.149128 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.150103 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.150083 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.150181 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.150112 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.150181 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.150126 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.150915 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.150834 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-166.ec2.internal.18b7679a8db7731f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-166.ec2.internal,UID:ip-10-0-139-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-166.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-166.ec2.internal,},FirstTimestamp:2026-06-09 11:54:51.148555039 +0000 UTC m=+0.430372256,LastTimestamp:2026-06-09 11:54:51.148555039 +0000 UTC m=+0.430372256,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-166.ec2.internal,}" Jun 09 11:54:51.151248 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.151236 2550 policy_none.go:49] "None policy: Start" Jun 09 11:54:51.151305 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.151253 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 11:54:51.151305 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.151265 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 11:54:51.152990 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.152978 2550 policy_none.go:47] "Start" Jun 09 11:54:51.158435 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.158421 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ks8d7" Jun 09 11:54:51.159552 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.159490 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-166.ec2.internal.18b7679a8db7d836 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-166.ec2.internal,UID:ip-10-0-139-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-166.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-166.ec2.internal,},FirstTimestamp:2026-06-09 11:54:51.148580918 +0000 UTC m=+0.430398135,LastTimestamp:2026-06-09 11:54:51.148580918 +0000 UTC m=+0.430398135,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-166.ec2.internal,}" Jun 09 11:54:51.165654 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.165636 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ks8d7" Jun 09 11:54:51.169067 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.169011 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-166.ec2.internal.18b7679a8db81450 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-166.ec2.internal,UID:ip-10-0-139-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-166.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-166.ec2.internal,},FirstTimestamp:2026-06-09 11:54:51.148596304 +0000 UTC m=+0.430413520,LastTimestamp:2026-06-09 11:54:51.148596304 +0000 UTC m=+0.430413520,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-166.ec2.internal,}" Jun 09 11:54:51.187281 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187261 2550 manager.go:337] "Starting Device Plugin manager" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.187291 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187301 2550 server.go:87] "Starting device plugin registration server" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187537 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187549 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187653 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187717 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.187725 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.188218 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.188268 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.195367 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.195402 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.195421 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.195485 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 11:54:51.207162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.199046 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:54:51.288539 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.288489 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.289268 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.289246 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.289354 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.289280 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.289354 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.289295 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.289354 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.289323 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.295718 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.295693 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal"] Jun 09 11:54:51.295788 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.295779 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.296436 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.296423 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.296506 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.296450 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.296506 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.296464 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.298841 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.298827 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.298936 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.298923 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.298978 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.298950 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.299444 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.299428 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.299510 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.299457 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.299510 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.299487 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.299510 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.299435 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.299694 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.299524 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.299694 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.299539 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.300654 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.300636 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.300686 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.300664 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-166.ec2.internal\": node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.301999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.301983 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.302076 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.302010 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:54:51.302611 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.302598 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:54:51.302690 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.302626 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:54:51.302690 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.302639 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:54:51.320280 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.320257 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-166.ec2.internal\" not found" node="ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.324410 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.324392 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-166.ec2.internal\" not found" node="ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.345003 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.344979 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.428277 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.428248 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a38314d8b55b654b06aedba48003627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal\" (UID: \"7a38314d8b55b654b06aedba48003627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.428277 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.428276 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a38314d8b55b654b06aedba48003627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal\" (UID: \"7a38314d8b55b654b06aedba48003627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.428406 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.428293 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d847be753db4224a797cf6deaab03dc-config\") pod \"kube-apiserver-proxy-ip-10-0-139-166.ec2.internal\" (UID: \"8d847be753db4224a797cf6deaab03dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.445618 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.445585 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.528400 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.528370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a38314d8b55b654b06aedba48003627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal\" (UID: \"7a38314d8b55b654b06aedba48003627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.528400 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.528400 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a38314d8b55b654b06aedba48003627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal\" (UID: \"7a38314d8b55b654b06aedba48003627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.528583 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.528418 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d847be753db4224a797cf6deaab03dc-config\") pod \"kube-apiserver-proxy-ip-10-0-139-166.ec2.internal\" (UID: \"8d847be753db4224a797cf6deaab03dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.528583 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.528459 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a38314d8b55b654b06aedba48003627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal\" (UID: \"7a38314d8b55b654b06aedba48003627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.528583 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.528461 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a38314d8b55b654b06aedba48003627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal\" (UID: \"7a38314d8b55b654b06aedba48003627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.528583 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.528492 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d847be753db4224a797cf6deaab03dc-config\") pod \"kube-apiserver-proxy-ip-10-0-139-166.ec2.internal\" (UID: \"8d847be753db4224a797cf6deaab03dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.546534 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.546472 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.626944 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.626918 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.631987 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.631539 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:51.646791 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.646773 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.747323 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.747285 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.847830 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.847759 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.948417 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:51.948380 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-166.ec2.internal\" not found" Jun 09 11:54:51.971008 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:51.970981 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:54:52.027117 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.027089 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" Jun 09 11:54:52.049367 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.049347 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 11:54:52.051362 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.051346 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:52.063030 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.063013 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 11:54:52.063132 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.063115 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 11:54:52.063187 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.063164 2550 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://af81ff03c7d2140bf80d0d98b90480e4-19e5d5851ffe75fa.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.139.166:44448->98.83.126.49:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" Jun 09 11:54:52.106281 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.106229 2550 apiserver.go:52] "Watching apiserver" Jun 09 11:54:52.120576 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.120550 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 11:54:52.122455 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.122427 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-4nd6j","openshift-ovn-kubernetes/ovnkube-node-gf7zd","kube-system/konnectivity-agent-5d8h9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5","openshift-cluster-node-tuning-operator/tuned-428kc","openshift-image-registry/node-ca-kzjmq","openshift-multus/multus-zxl94","openshift-multus/network-metrics-daemon-j8vvj","kube-system/global-pull-secret-syncer-knwgt","kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal","openshift-dns/node-resolver-p9fst","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal","openshift-multus/multus-additional-cni-plugins-7cnfm","openshift-network-diagnostics/network-check-target-j8m4f"] Jun 09 11:54:52.124841 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.124825 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.126729 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.126706 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 11:54:52.127433 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.127411 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.127968 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.127952 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.128390 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.128373 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 11:54:52.128770 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.128752 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.128847 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.128754 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 11:54:52.129238 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.129222 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 11:54:52.129455 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.129442 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.130798 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130779 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.130888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130803 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 11:54:52.130888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130810 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98t8\" (UniqueName: \"kubernetes.io/projected/ea6ed8f4-01b5-484a-af18-20b31c632acd-kube-api-access-v98t8\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.130888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130839 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-system-cni-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.130888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130863 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-k8s-cni-cncf-io\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131037 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130911 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-cni-multus\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131037 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130934 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2jt7x\"" Jun 09 11:54:52.131037 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130952 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-system-cni-dir\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.131037 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130973 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.131037 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.130990 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-cni-bin\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131037 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131018 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-daemon-config\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131040 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-etc-kubernetes\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131061 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131076 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-cni-binary-copy\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131089 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-cni-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131106 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-os-release\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131126 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-kubelet\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131141 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e3219bd-6dce-4959-ace5-290c62ed8a76-konnectivity-ca\") pod \"konnectivity-agent-5d8h9\" (UID: \"7e3219bd-6dce-4959-ace5-290c62ed8a76\") " pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131163 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-hostroot\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131187 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-cnibin\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.131212 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131208 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-os-release\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131223 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-conf-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131245 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-multus-certs\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131264 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131278 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e3219bd-6dce-4959-ace5-290c62ed8a76-agent-certs\") pod \"konnectivity-agent-5d8h9\" (UID: \"7e3219bd-6dce-4959-ace5-290c62ed8a76\") " pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131292 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-socket-dir-parent\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131309 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-netns\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131340 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-cnibin\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131356 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr9s6\" (UniqueName: \"kubernetes.io/projected/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-kube-api-access-sr9s6\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.131827 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.131809 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.133715 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.133699 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.135717 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.135671 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.135893 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.135875 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zvfqq\"" Jun 09 11:54:52.136014 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.135997 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 11:54:52.136014 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136010 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 11:54:52.136197 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136178 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.136284 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136267 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.136453 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136429 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-78p8s\"" Jun 09 11:54:52.136520 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136465 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 11:54:52.136520 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136469 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8mzfd\"" Jun 09 11:54:52.136622 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136537 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.136622 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136537 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.136622 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.136603 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-89hq8\"" Jun 09 11:54:52.138023 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.138008 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.138101 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.138080 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:54:52.140055 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.140030 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.142207 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.142192 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.144555 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.144539 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:52.144629 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.144586 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:54:52.145981 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.145959 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.146066 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.145999 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.146066 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146028 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.146066 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146042 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.146208 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146091 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 11:54:52.146208 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146132 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 11:54:52.146309 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146241 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.146421 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146401 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lzw46\"" Jun 09 11:54:52.146590 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146578 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 11:54:52.146644 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146630 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 11:54:52.146688 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146666 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k7wwn\"" Jun 09 11:54:52.146723 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146716 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 11:54:52.146860 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146848 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 11:54:52.146860 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146850 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vqqgm\"" Jun 09 11:54:52.146964 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.146871 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.147281 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.147263 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.149437 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.149423 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.149499 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.149470 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:54:52.157236 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.157219 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 11:54:52.157310 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.157267 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 11:54:52.157357 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.157339 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mcsvb\"" Jun 09 11:54:52.157724 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.157710 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 11:54:52.170726 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.168809 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 11:49:51 +0000 UTC" deadline="2027-11-18 05:02:46.708844304 +0000 UTC" Jun 09 11:54:52.170726 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.168879 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12641h7m54.539970583s" Jun 09 11:54:52.187782 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.187765 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k56xz" Jun 09 11:54:52.208863 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.208644 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k56xz" Jun 09 11:54:52.216970 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.216950 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d847be753db4224a797cf6deaab03dc.slice/crio-c0fcea5975f28ba3864271ca028574a83cef02c90ae3d57fac27c7326c7d833f WatchSource:0}: Error finding container c0fcea5975f28ba3864271ca028574a83cef02c90ae3d57fac27c7326c7d833f: Status 404 returned error can't find the container with id c0fcea5975f28ba3864271ca028574a83cef02c90ae3d57fac27c7326c7d833f Jun 09 11:54:52.217239 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.217221 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a38314d8b55b654b06aedba48003627.slice/crio-454341e5f3ca2939c4634cf77f8d469c6ddfe14a78ffcc723192827e2ee20831 WatchSource:0}: Error finding container 454341e5f3ca2939c4634cf77f8d469c6ddfe14a78ffcc723192827e2ee20831: Status 404 returned error can't find the container with id 454341e5f3ca2939c4634cf77f8d469c6ddfe14a78ffcc723192827e2ee20831 Jun 09 11:54:52.221911 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.221896 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 11:54:52.228024 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.228001 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 11:54:52.231785 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231763 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-os-release\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.231888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231801 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-registration-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.231888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231825 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysctl-d\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.231888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231848 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-systemd\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.231888 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231872 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02290cca-3ae9-4935-bac3-f1da28c74837-etc-tuned\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.232064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231909 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-os-release\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.232064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231939 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-slash\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.232064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.231980 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.232064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232007 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxb7h\" (UniqueName: \"kubernetes.io/projected/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-kube-api-access-qxb7h\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.232064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232043 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e3219bd-6dce-4959-ace5-290c62ed8a76-agent-certs\") pod \"konnectivity-agent-5d8h9\" (UID: \"7e3219bd-6dce-4959-ace5-290c62ed8a76\") " pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232066 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-socket-dir-parent\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232091 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-netns\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232145 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-netns\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232160 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-socket-dir-parent\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-cnibin\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232229 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr9s6\" (UniqueName: \"kubernetes.io/projected/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-kube-api-access-sr9s6\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232254 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-sys-fs\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232272 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-cnibin\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232278 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp9tq\" (UniqueName: \"kubernetes.io/projected/c82f0257-6b8d-493f-ad4f-ff3411eabc27-kube-api-access-dp9tq\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232303 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-modprobe-d\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232324 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-host\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232344 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-var-lib-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232369 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovn-node-metrics-cert\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232345 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232395 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232419 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v98t8\" (UniqueName: \"kubernetes.io/projected/ea6ed8f4-01b5-484a-af18-20b31c632acd-kube-api-access-v98t8\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232441 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-k8s-cni-cncf-io\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232464 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-cni-multus\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232514 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-cni-multus\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232526 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-k8s-cni-cncf-io\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.232569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232564 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232621 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232643 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8888m\" (UniqueName: \"kubernetes.io/projected/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-kube-api-access-8888m\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232666 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-node-log\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232693 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5m7z\" (UniqueName: \"kubernetes.io/projected/de637c06-0362-4c2d-b07f-af51cc52e6ce-kube-api-access-g5m7z\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232748 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-cni-bin\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232792 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-etc-kubernetes\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232841 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-cni-bin\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232844 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-systemd-units\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232874 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-etc-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232897 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-env-overrides\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232905 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-etc-kubernetes\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232921 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovnkube-script-lib\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232950 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-tmp-dir\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232972 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.232989 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-cni-binary-copy\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233004 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2098e65-cf15-4be3-b98c-4afe77fee72c-kubelet-config\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.233250 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233019 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-lib-modules\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233034 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-systemd\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233049 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-socket-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233062 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2098e65-cf15-4be3-b98c-4afe77fee72c-dbus\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233078 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c82f0257-6b8d-493f-ad4f-ff3411eabc27-iptables-alerter-script\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233100 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysconfig\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233124 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-kubernetes\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233146 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-cni-netd\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233169 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-host\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233188 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-serviceca\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233206 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233264 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-sys\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233325 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-var-lib-kubelet\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233341 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-cnibin\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233354 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-conf-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233369 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-multus-certs\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233395 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-ovn\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.234214 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233418 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233442 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-device-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233469 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233486 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-cnibin\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233492 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysctl-conf\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233522 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-run-netns\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233525 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-run-multus-certs\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233545 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-cni-binary-copy\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233446 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-conf-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233547 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233679 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-kubelet\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-system-cni-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233757 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hs5z\" (UniqueName: \"kubernetes.io/projected/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-kube-api-access-6hs5z\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233788 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02290cca-3ae9-4935-bac3-f1da28c74837-tmp\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233788 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233815 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233833 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-system-cni-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233862 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-system-cni-dir\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233888 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233911 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-daemon-config\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233918 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea6ed8f4-01b5-484a-af18-20b31c632acd-system-cni-dir\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233934 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-run\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.233996 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234033 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-log-socket\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234061 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llfkw\" (UniqueName: \"kubernetes.io/projected/02290cca-3ae9-4935-bac3-f1da28c74837-kube-api-access-llfkw\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234087 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-cni-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234149 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-os-release\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234172 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-cni-dir\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234184 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-kubelet\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234287 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234228 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-os-release\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234339 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cctx4\" (UniqueName: \"kubernetes.io/projected/d32b83f1-94cf-4a81-9f0b-7d4afac31203-kube-api-access-cctx4\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234356 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea6ed8f4-01b5-484a-af18-20b31c632acd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234374 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-hosts-file\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.235672 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234236 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-host-var-lib-kubelet\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234403 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-multus-daemon-config\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234412 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e3219bd-6dce-4959-ace5-290c62ed8a76-konnectivity-ca\") pod \"konnectivity-agent-5d8h9\" (UID: \"7e3219bd-6dce-4959-ace5-290c62ed8a76\") " pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234439 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-hostroot\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234465 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c82f0257-6b8d-493f-ad4f-ff3411eabc27-host-slash\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-cni-bin\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234536 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234511 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-hostroot\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234578 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovnkube-config\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.234986 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e3219bd-6dce-4959-ace5-290c62ed8a76-konnectivity-ca\") pod \"konnectivity-agent-5d8h9\" (UID: \"7e3219bd-6dce-4959-ace5-290c62ed8a76\") " pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.236228 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.235592 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e3219bd-6dce-4959-ace5-290c62ed8a76-agent-certs\") pod \"konnectivity-agent-5d8h9\" (UID: \"7e3219bd-6dce-4959-ace5-290c62ed8a76\") " pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.246826 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.246805 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr9s6\" (UniqueName: \"kubernetes.io/projected/b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e-kube-api-access-sr9s6\") pod \"multus-zxl94\" (UID: \"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e\") " pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.248210 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.248195 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98t8\" (UniqueName: \"kubernetes.io/projected/ea6ed8f4-01b5-484a-af18-20b31c632acd-kube-api-access-v98t8\") pod \"multus-additional-cni-plugins-7cnfm\" (UID: \"ea6ed8f4-01b5-484a-af18-20b31c632acd\") " pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.334971 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.334944 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-systemd-units\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335124 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.334980 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-etc-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335124 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.334996 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-env-overrides\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335124 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335010 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovnkube-script-lib\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335124 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335060 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-etc-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335263 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335187 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-systemd-units\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335263 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335218 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-tmp-dir\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.335263 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335245 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2098e65-cf15-4be3-b98c-4afe77fee72c-kubelet-config\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.335410 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335269 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-lib-modules\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.335410 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335292 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-systemd\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335410 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335335 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-systemd\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335410 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335352 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2098e65-cf15-4be3-b98c-4afe77fee72c-kubelet-config\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.335410 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335393 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-socket-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335420 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2098e65-cf15-4be3-b98c-4afe77fee72c-dbus\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335443 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c82f0257-6b8d-493f-ad4f-ff3411eabc27-iptables-alerter-script\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335461 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-lib-modules\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335469 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysconfig\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335487 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-env-overrides\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335505 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-kubernetes\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335533 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-tmp-dir\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335531 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-cni-netd\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335558 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysconfig\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335507 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-socket-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335581 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-host\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335606 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-serviceca\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335610 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-kubernetes\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335625 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2098e65-cf15-4be3-b98c-4afe77fee72c-dbus\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335634 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-cni-netd\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.335649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335641 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-host\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335659 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovnkube-script-lib\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335690 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-sys\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335759 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-var-lib-kubelet\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335786 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-sys\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335790 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-ovn\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335816 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-device-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335826 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-var-lib-kubelet\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335839 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335855 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-device-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335856 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-ovn\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335863 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysctl-conf\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335891 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335889 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-run-netns\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335924 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-run-netns\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335926 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.336274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335941 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-serviceca\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.335966 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-kubelet\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336019 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysctl-conf\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336042 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hs5z\" (UniqueName: \"kubernetes.io/projected/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-kube-api-access-6hs5z\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336048 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-kubelet\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.336018 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336065 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02290cca-3ae9-4935-bac3-f1da28c74837-tmp\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.336106 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:54:52.836091198 +0000 UTC m=+2.117908404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336184 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336218 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-run\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336240 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-log-socket\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336265 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llfkw\" (UniqueName: \"kubernetes.io/projected/02290cca-3ae9-4935-bac3-f1da28c74837-kube-api-access-llfkw\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336271 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-run\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336273 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336298 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-log-socket\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336316 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336342 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cctx4\" (UniqueName: \"kubernetes.io/projected/d32b83f1-94cf-4a81-9f0b-7d4afac31203-kube-api-access-cctx4\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336378 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-hosts-file\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336396 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c82f0257-6b8d-493f-ad4f-ff3411eabc27-host-slash\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.336410 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.336459 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:54:52.836443027 +0000 UTC m=+2.118260232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336462 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-hosts-file\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336478 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-cni-bin\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336501 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336517 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovnkube-config\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336516 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-cni-bin\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336480 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c82f0257-6b8d-493f-ad4f-ff3411eabc27-host-slash\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336539 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-registration-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336565 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336584 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-registration-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336598 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysctl-d\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336649 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-systemd\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336698 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-systemd\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336704 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02290cca-3ae9-4935-bac3-f1da28c74837-etc-tuned\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.337659 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336709 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c82f0257-6b8d-493f-ad4f-ff3411eabc27-iptables-alerter-script\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336779 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-sysctl-d\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-slash\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336847 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336863 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxb7h\" (UniqueName: \"kubernetes.io/projected/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-kube-api-access-qxb7h\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336880 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-sys-fs\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336896 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp9tq\" (UniqueName: \"kubernetes.io/projected/c82f0257-6b8d-493f-ad4f-ff3411eabc27-kube-api-access-dp9tq\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336916 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-modprobe-d\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336920 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-run-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336934 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-host\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336952 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-var-lib-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336959 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-host-slash\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.336966 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovn-node-metrics-cert\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337022 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovnkube-config\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337043 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-etc-modprobe-d\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337052 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-sys-fs\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337072 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.338241 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337062 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02290cca-3ae9-4935-bac3-f1da28c74837-host\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.338722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337177 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8888m\" (UniqueName: \"kubernetes.io/projected/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-kube-api-access-8888m\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.338722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337202 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.338722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337276 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-var-lib-openvswitch\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337367 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-node-log\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337675 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d32b83f1-94cf-4a81-9f0b-7d4afac31203-node-log\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.338722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.337713 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5m7z\" (UniqueName: \"kubernetes.io/projected/de637c06-0362-4c2d-b07f-af51cc52e6ce-kube-api-access-g5m7z\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.338909 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.338759 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02290cca-3ae9-4935-bac3-f1da28c74837-etc-tuned\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.338909 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.338841 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02290cca-3ae9-4935-bac3-f1da28c74837-tmp\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.339083 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.339068 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d32b83f1-94cf-4a81-9f0b-7d4afac31203-ovn-node-metrics-cert\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.350196 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.350176 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:54:52.350196 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.350197 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:54:52.350356 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.350210 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:52.350356 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.350266 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:54:52.850250375 +0000 UTC m=+2.132067595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:52.350849 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.350541 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llfkw\" (UniqueName: \"kubernetes.io/projected/02290cca-3ae9-4935-bac3-f1da28c74837-kube-api-access-llfkw\") pod \"tuned-428kc\" (UID: \"02290cca-3ae9-4935-bac3-f1da28c74837\") " pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.350849 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.350816 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp9tq\" (UniqueName: \"kubernetes.io/projected/c82f0257-6b8d-493f-ad4f-ff3411eabc27-kube-api-access-dp9tq\") pod \"iptables-alerter-4nd6j\" (UID: \"c82f0257-6b8d-493f-ad4f-ff3411eabc27\") " pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.350958 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.350875 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hs5z\" (UniqueName: \"kubernetes.io/projected/0d0a2e7e-9116-4e14-a83e-2708fd94c97b-kube-api-access-6hs5z\") pod \"node-ca-kzjmq\" (UID: \"0d0a2e7e-9116-4e14-a83e-2708fd94c97b\") " pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.351917 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.351898 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8888m\" (UniqueName: \"kubernetes.io/projected/03dd62b7-4577-4874-b4c2-2cc71ae50bbd-kube-api-access-8888m\") pod \"aws-ebs-csi-driver-node-f8zj5\" (UID: \"03dd62b7-4577-4874-b4c2-2cc71ae50bbd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.352418 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.352403 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cctx4\" (UniqueName: \"kubernetes.io/projected/d32b83f1-94cf-4a81-9f0b-7d4afac31203-kube-api-access-cctx4\") pod \"ovnkube-node-gf7zd\" (UID: \"d32b83f1-94cf-4a81-9f0b-7d4afac31203\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.352649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.352631 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5m7z\" (UniqueName: \"kubernetes.io/projected/de637c06-0362-4c2d-b07f-af51cc52e6ce-kube-api-access-g5m7z\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.352987 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.352971 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxb7h\" (UniqueName: \"kubernetes.io/projected/f55d0bdb-53c4-4e01-b6e3-be9388a0de06-kube-api-access-qxb7h\") pod \"node-resolver-p9fst\" (UID: \"f55d0bdb-53c4-4e01-b6e3-be9388a0de06\") " pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.457121 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.457097 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxl94" Jun 09 11:54:52.466378 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.466357 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f81014_5a96_4c05_9c6d_0b6f6fa2d90e.slice/crio-1edaf6464f0026d732411f399fd0a960005cafdc8d215041d1d03100d2b96c82 WatchSource:0}: Error finding container 1edaf6464f0026d732411f399fd0a960005cafdc8d215041d1d03100d2b96c82: Status 404 returned error can't find the container with id 1edaf6464f0026d732411f399fd0a960005cafdc8d215041d1d03100d2b96c82 Jun 09 11:54:52.474330 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.474313 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" Jun 09 11:54:52.481395 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.481377 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:54:52.481985 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.481962 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6ed8f4_01b5_484a_af18_20b31c632acd.slice/crio-827a31ce54e35acb2236f22ecc50f0a0da0f46813b05c78d679e4c268a34ef1c WatchSource:0}: Error finding container 827a31ce54e35acb2236f22ecc50f0a0da0f46813b05c78d679e4c268a34ef1c: Status 404 returned error can't find the container with id 827a31ce54e35acb2236f22ecc50f0a0da0f46813b05c78d679e4c268a34ef1c Jun 09 11:54:52.490763 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.490725 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3219bd_6dce_4959_ace5_290c62ed8a76.slice/crio-8442658edcf7bc0cb6fc8a3a3bebd593ba6a7af6a3eb126e6556efca5e00e965 WatchSource:0}: Error finding container 8442658edcf7bc0cb6fc8a3a3bebd593ba6a7af6a3eb126e6556efca5e00e965: Status 404 returned error can't find the container with id 8442658edcf7bc0cb6fc8a3a3bebd593ba6a7af6a3eb126e6556efca5e00e965 Jun 09 11:54:52.497632 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.497617 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" Jun 09 11:54:52.505064 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.505043 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03dd62b7_4577_4874_b4c2_2cc71ae50bbd.slice/crio-1ce43d49653df2e8113b1b8372b2c5a7a88f6440d735606491c000d5f1b2ea19 WatchSource:0}: Error finding container 1ce43d49653df2e8113b1b8372b2c5a7a88f6440d735606491c000d5f1b2ea19: Status 404 returned error can't find the container with id 1ce43d49653df2e8113b1b8372b2c5a7a88f6440d735606491c000d5f1b2ea19 Jun 09 11:54:52.517707 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.517686 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-428kc" Jun 09 11:54:52.523685 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.523665 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kzjmq" Jun 09 11:54:52.524665 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.524645 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02290cca_3ae9_4935_bac3_f1da28c74837.slice/crio-7412cb8c93404482de3930c7cb5ed817699189b5b5b6843c28d98f3556830cf1 WatchSource:0}: Error finding container 7412cb8c93404482de3930c7cb5ed817699189b5b5b6843c28d98f3556830cf1: Status 404 returned error can't find the container with id 7412cb8c93404482de3930c7cb5ed817699189b5b5b6843c28d98f3556830cf1 Jun 09 11:54:52.529858 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.529842 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4nd6j" Jun 09 11:54:52.530989 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.530969 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0a2e7e_9116_4e14_a83e_2708fd94c97b.slice/crio-a1dbf7b0bce97cbc51c941f98e5e713e6cf600fe022ef0896bb746782c3f27b8 WatchSource:0}: Error finding container a1dbf7b0bce97cbc51c941f98e5e713e6cf600fe022ef0896bb746782c3f27b8: Status 404 returned error can't find the container with id a1dbf7b0bce97cbc51c941f98e5e713e6cf600fe022ef0896bb746782c3f27b8 Jun 09 11:54:52.537222 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.537202 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:54:52.539306 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.539287 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82f0257_6b8d_493f_ad4f_ff3411eabc27.slice/crio-d710e665e69eb0a5419a834763e50683fe68b91e07ef874eb7d81cccb97f24fa WatchSource:0}: Error finding container d710e665e69eb0a5419a834763e50683fe68b91e07ef874eb7d81cccb97f24fa: Status 404 returned error can't find the container with id d710e665e69eb0a5419a834763e50683fe68b91e07ef874eb7d81cccb97f24fa Jun 09 11:54:52.545179 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.545160 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p9fst" Jun 09 11:54:52.545386 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.545369 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32b83f1_94cf_4a81_9f0b_7d4afac31203.slice/crio-0895d649439815b17701c172c3af4b750fb69eda8621758804cd56705b4a8085 WatchSource:0}: Error finding container 0895d649439815b17701c172c3af4b750fb69eda8621758804cd56705b4a8085: Status 404 returned error can't find the container with id 0895d649439815b17701c172c3af4b750fb69eda8621758804cd56705b4a8085 Jun 09 11:54:52.553440 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:54:52.553421 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55d0bdb_53c4_4e01_b6e3_be9388a0de06.slice/crio-c651923f4f968ce135eddb719f4ba33a415c9202fb8841378857c17bdf2b4296 WatchSource:0}: Error finding container c651923f4f968ce135eddb719f4ba33a415c9202fb8841378857c17bdf2b4296: Status 404 returned error can't find the container with id c651923f4f968ce135eddb719f4ba33a415c9202fb8841378857c17bdf2b4296 Jun 09 11:54:52.590086 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.590062 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:54:52.733529 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.733457 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:54:52.841269 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.841231 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:52.841436 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.841293 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:52.841492 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.841436 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:52.841543 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.841497 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:54:53.841480476 +0000 UTC m=+3.123297693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:52.841936 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.841908 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:52.842053 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.841970 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:54:53.841952618 +0000 UTC m=+3.123769819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:52.942762 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:52.942714 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:52.942956 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.942937 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:54:52.943026 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.942962 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:54:52.943026 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.942975 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:52.943131 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:52.943030 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:54:53.943012432 +0000 UTC m=+3.224829646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:53.125558 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.125529 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:54:53.198498 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.198049 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:53.198498 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.198177 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:54:53.212080 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.211996 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p9fst" event={"ID":"f55d0bdb-53c4-4e01-b6e3-be9388a0de06","Type":"ContainerStarted","Data":"c651923f4f968ce135eddb719f4ba33a415c9202fb8841378857c17bdf2b4296"} Jun 09 11:54:53.212569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.212491 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 11:49:52 +0000 UTC" deadline="2027-12-21 20:11:27.148911208 +0000 UTC" Jun 09 11:54:53.212569 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.212528 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13448h16m33.936387573s" Jun 09 11:54:53.230522 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.229943 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"0895d649439815b17701c172c3af4b750fb69eda8621758804cd56705b4a8085"} Jun 09 11:54:53.240867 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.240839 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4nd6j" event={"ID":"c82f0257-6b8d-493f-ad4f-ff3411eabc27","Type":"ContainerStarted","Data":"d710e665e69eb0a5419a834763e50683fe68b91e07ef874eb7d81cccb97f24fa"} Jun 09 11:54:53.265043 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.264980 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-428kc" event={"ID":"02290cca-3ae9-4935-bac3-f1da28c74837","Type":"ContainerStarted","Data":"7412cb8c93404482de3930c7cb5ed817699189b5b5b6843c28d98f3556830cf1"} Jun 09 11:54:53.284660 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.284633 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" event={"ID":"03dd62b7-4577-4874-b4c2-2cc71ae50bbd","Type":"ContainerStarted","Data":"1ce43d49653df2e8113b1b8372b2c5a7a88f6440d735606491c000d5f1b2ea19"} Jun 09 11:54:53.309959 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.309868 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5d8h9" event={"ID":"7e3219bd-6dce-4959-ace5-290c62ed8a76","Type":"ContainerStarted","Data":"8442658edcf7bc0cb6fc8a3a3bebd593ba6a7af6a3eb126e6556efca5e00e965"} Jun 09 11:54:53.328194 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.328165 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxl94" event={"ID":"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e","Type":"ContainerStarted","Data":"1edaf6464f0026d732411f399fd0a960005cafdc8d215041d1d03100d2b96c82"} Jun 09 11:54:53.341064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.341038 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kzjmq" event={"ID":"0d0a2e7e-9116-4e14-a83e-2708fd94c97b","Type":"ContainerStarted","Data":"a1dbf7b0bce97cbc51c941f98e5e713e6cf600fe022ef0896bb746782c3f27b8"} Jun 09 11:54:53.362803 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.362771 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerStarted","Data":"827a31ce54e35acb2236f22ecc50f0a0da0f46813b05c78d679e4c268a34ef1c"} Jun 09 11:54:53.373634 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.373605 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" event={"ID":"7a38314d8b55b654b06aedba48003627","Type":"ContainerStarted","Data":"454341e5f3ca2939c4634cf77f8d469c6ddfe14a78ffcc723192827e2ee20831"} Jun 09 11:54:53.411838 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.411777 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" event={"ID":"8d847be753db4224a797cf6deaab03dc","Type":"ContainerStarted","Data":"c0fcea5975f28ba3864271ca028574a83cef02c90ae3d57fac27c7326c7d833f"} Jun 09 11:54:53.849224 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.849188 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:53.849393 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.849246 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:53.849393 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.849379 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:53.849510 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.849442 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:54:55.849422967 +0000 UTC m=+5.131240168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:53.849863 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.849843 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:53.849955 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.849897 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:54:55.849882457 +0000 UTC m=+5.131699662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:53.949861 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:53.949824 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:53.950049 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.950028 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:54:53.950117 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.950059 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:54:53.950117 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.950072 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:53.950214 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:53.950129 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:54:55.950110873 +0000 UTC m=+5.231928090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:54.196374 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:54.196345 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:54.196854 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:54.196464 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:54:54.196854 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:54.196533 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:54.196854 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:54.196600 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:54:54.212817 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:54.212754 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 11:49:52 +0000 UTC" deadline="2028-02-09 09:50:39.942899018 +0000 UTC" Jun 09 11:54:54.212817 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:54.212815 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14637h55m45.730088181s" Jun 09 11:54:55.198513 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:55.198483 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:55.198983 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.198620 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:54:55.864304 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:55.864273 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:55.864473 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:55.864314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:55.864473 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.864426 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:55.864473 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.864473 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:54:59.864459261 +0000 UTC m=+9.146276462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:55.864841 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.864822 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:55.864917 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.864869 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:54:59.864857736 +0000 UTC m=+9.146674938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:55.965041 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:55.965007 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:55.965223 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.965202 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:54:55.965313 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.965229 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:54:55.965313 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.965242 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:55.965313 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:55.965297 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:54:59.965280284 +0000 UTC m=+9.247097504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:54:56.196065 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:56.196036 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:56.196251 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:56.196036 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:56.196328 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:56.196262 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:54:56.196328 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:56.196167 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:54:57.197345 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:57.196842 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:57.197345 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:57.196978 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:54:58.196199 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:58.196109 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:58.196199 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:58.196128 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:54:58.196430 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:58.196408 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:54:58.196496 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:58.196247 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:54:59.196122 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:59.196091 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:59.196699 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:59.196237 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:54:59.899829 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:59.899788 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:54:59.900013 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:54:59.899836 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:54:59.900013 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:59.899951 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:59.900124 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:59.900014 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:54:59.900124 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:59.900035 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:55:07.900016272 +0000 UTC m=+17.181833490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:54:59.900236 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:54:59.900133 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:55:07.900115233 +0000 UTC m=+17.181932450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:55:00.000313 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:00.000274 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:00.000525 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:00.000496 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:55:00.000616 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:00.000529 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:55:00.000616 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:00.000546 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:55:00.000616 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:00.000605 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:55:08.000587058 +0000 UTC m=+17.282404284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:55:00.196585 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:00.195930 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:00.196585 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:00.196046 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:00.196585 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:00.196404 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:00.196585 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:00.196492 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:01.200070 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:01.200038 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:01.200513 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:01.200188 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:02.196404 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:02.196336 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:02.196581 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:02.196466 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:02.196581 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:02.196521 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:02.196697 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:02.196652 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:03.196323 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:03.196290 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:03.196773 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:03.196439 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:04.195927 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:04.195850 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:04.195927 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:04.195886 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:04.196137 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:04.195970 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:04.196137 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:04.196116 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:05.196429 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:05.196399 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:05.196925 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:05.196507 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:06.196591 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:06.196559 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:06.197059 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:06.196680 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:06.197059 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:06.196811 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:06.197059 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:06.196918 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:07.196253 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:07.196218 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:07.196423 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:07.196357 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:07.963717 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:07.963687 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:07.964134 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:07.963725 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:07.964134 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:07.963838 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:55:07.964134 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:07.963848 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:55:07.964134 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:07.963885 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:55:23.96387242 +0000 UTC m=+33.245689621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:55:07.964134 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:07.963909 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:55:23.963891231 +0000 UTC m=+33.245708432 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:55:08.064479 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:08.064448 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:08.064655 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:08.064635 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:55:08.064721 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:08.064665 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:55:08.064721 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:08.064679 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:55:08.064836 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:08.064758 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:55:24.064721233 +0000 UTC m=+33.346538449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:55:08.195674 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:08.195642 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:08.195674 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:08.195666 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:08.195909 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:08.195798 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:08.195972 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:08.195948 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:09.195819 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:09.195785 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:09.196233 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:09.195921 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:10.196307 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:10.196277 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:10.196756 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:10.196277 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:10.196756 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:10.196374 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:10.196756 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:10.196478 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:11.196886 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:11.196856 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:11.197345 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:11.196963 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:12.195954 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:12.195927 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:12.196223 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:12.195932 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:12.196223 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:12.196012 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:12.196223 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:12.196102 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:13.196154 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.195721 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:13.196907 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:13.196280 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:13.461026 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.460993 2550 generic.go:356] "Generic (PLEG): container finished" podID="ea6ed8f4-01b5-484a-af18-20b31c632acd" containerID="5ccb5ec804d58400835f7dca09f91e16d5c7969c592e7915306cc3348a36012e" exitCode=0 Jun 09 11:55:13.461148 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.461080 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerDied","Data":"5ccb5ec804d58400835f7dca09f91e16d5c7969c592e7915306cc3348a36012e"} Jun 09 11:55:13.462658 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.462633 2550 generic.go:356] "Generic (PLEG): container finished" podID="7a38314d8b55b654b06aedba48003627" containerID="8d66e7be4507778210f2fdb45a1e726bf6e6d2e48e841bca954dbfcaf597e730" exitCode=0 Jun 09 11:55:13.462765 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.462724 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" event={"ID":"7a38314d8b55b654b06aedba48003627","Type":"ContainerDied","Data":"8d66e7be4507778210f2fdb45a1e726bf6e6d2e48e841bca954dbfcaf597e730"} Jun 09 11:55:13.464646 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.464613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" event={"ID":"8d847be753db4224a797cf6deaab03dc","Type":"ContainerStarted","Data":"1b4a6e5a4fded4bc055f5ce3ccc6dfbb77d47d7a646266bdbe31192c06429fe1"} Jun 09 11:55:13.466142 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.466118 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p9fst" event={"ID":"f55d0bdb-53c4-4e01-b6e3-be9388a0de06","Type":"ContainerStarted","Data":"0b970c7730485883ef35a09777f508cae9a717d670989600171f338192da1c8a"} Jun 09 11:55:13.469211 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.469190 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"8ff2129ff6c5be1473f9542d8c7daaeb9cecbd4f35ff044f8125126c9c519faa"} Jun 09 11:55:13.469288 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.469246 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"f258d2a388958a697fe03e8375a1b3aece7a52713415e808789441e317c14511"} Jun 09 11:55:13.469288 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.469281 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"cbacb9496ad7e93cc1eab1771db390ad0706a989eec9544efe1de1eda199f073"} Jun 09 11:55:13.469357 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.469320 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"41a79f92cd4bce202054943161e6a6e255efd63460327cf8f57bfab535d7a8ac"} Jun 09 11:55:13.469357 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.469353 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"6e648c93f5fdbe9c2965565ac5644408c8540ce090a1fe7d2a758d5f99eb0378"} Jun 09 11:55:13.469416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.469405 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"3bd6ddd472d211343f911fb0352a1401463d88aa05a9459bc5dbedb3a90c00db"} Jun 09 11:55:13.470711 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.470688 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-428kc" event={"ID":"02290cca-3ae9-4935-bac3-f1da28c74837","Type":"ContainerStarted","Data":"242edb8e9c56dbcd6a418be06aa96f80ba96dde7d83f49edda28d90a7f59684f"} Jun 09 11:55:13.472257 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.472221 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" event={"ID":"03dd62b7-4577-4874-b4c2-2cc71ae50bbd","Type":"ContainerStarted","Data":"1867dea4614906b39f0036971ecc50a38dc6528e2947dd9931657ede26329e43"} Jun 09 11:55:13.473627 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.473599 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5d8h9" event={"ID":"7e3219bd-6dce-4959-ace5-290c62ed8a76","Type":"ContainerStarted","Data":"a5b9dfa08abfc0c79ee0b4ed7a2648a1f77071f280dc84f91157a53392613d34"} Jun 09 11:55:13.475005 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.474976 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxl94" event={"ID":"b3f81014-5a96-4c05-9c6d-0b6f6fa2d90e","Type":"ContainerStarted","Data":"ac03525e41dd8606feb921be81cb404b88c95cf6d7d32f9b0b47f9ae2ad09a8d"} Jun 09 11:55:13.478775 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.478752 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kzjmq" event={"ID":"0d0a2e7e-9116-4e14-a83e-2708fd94c97b","Type":"ContainerStarted","Data":"973c95e611b270b36c8cd00fa92f41a3d26eaf781da28176b73cfa319d9cc351"} Jun 09 11:55:13.509630 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.509569 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p9fst" podStartSLOduration=2.8154234259999997 podStartE2EDuration="22.50955058s" totalImagesPullingTime="19.694127154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:13.508849773 +0000 UTC m=+22.790666997" watchObservedRunningTime="2026-06-09 11:55:13.50955058 +0000 UTC m=+22.791367803" Jun 09 11:55:13.543230 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.543191 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5d8h9" podStartSLOduration=2.788214365 podStartE2EDuration="22.543177586s" totalImagesPullingTime="19.754963221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:13.542921352 +0000 UTC m=+22.824738586" watchObservedRunningTime="2026-06-09 11:55:13.543177586 +0000 UTC m=+22.824994808" Jun 09 11:55:13.560970 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.560792 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-428kc" podStartSLOduration=2.83701162 podStartE2EDuration="22.560778553s" totalImagesPullingTime="19.723766933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:13.559696509 +0000 UTC m=+22.841513734" watchObservedRunningTime="2026-06-09 11:55:13.560778553 +0000 UTC m=+22.842595773" Jun 09 11:55:13.576046 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.576012 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-166.ec2.internal" podStartSLOduration=21.57599811 podStartE2EDuration="21.57599811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:13.575998614 +0000 UTC m=+22.857815837" watchObservedRunningTime="2026-06-09 11:55:13.57599811 +0000 UTC m=+22.857815332" Jun 09 11:55:13.604643 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.604594 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zxl94" podStartSLOduration=2.789756729 podStartE2EDuration="22.60457457s" totalImagesPullingTime="19.814817841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:13.604376277 +0000 UTC m=+22.886193500" watchObservedRunningTime="2026-06-09 11:55:13.60457457 +0000 UTC m=+22.886391797" Jun 09 11:55:13.715600 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:13.715545 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kzjmq" podStartSLOduration=3.000837295 podStartE2EDuration="22.715526063s" totalImagesPullingTime="19.714688768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:13.708941459 +0000 UTC m=+22.990758683" watchObservedRunningTime="2026-06-09 11:55:13.715526063 +0000 UTC m=+22.997343285" Jun 09 11:55:14.196509 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.196482 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:14.196862 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:14.196578 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:14.196862 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.196641 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:14.196862 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:14.196777 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:14.313682 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.313655 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 11:55:14.482680 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.482605 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" event={"ID":"7a38314d8b55b654b06aedba48003627","Type":"ContainerStarted","Data":"41ae9dfcbf49f1d4b2cc13bb7cd6b202023c617677d08b6cac926ed62dea3977"} Jun 09 11:55:14.484446 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.484418 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4nd6j" event={"ID":"c82f0257-6b8d-493f-ad4f-ff3411eabc27","Type":"ContainerStarted","Data":"9a81604cfc5b575a007a32d16a80ae5770e1676b9f858c40c7aa0b11e983377d"} Jun 09 11:55:14.486482 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.486448 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" event={"ID":"03dd62b7-4577-4874-b4c2-2cc71ae50bbd","Type":"ContainerStarted","Data":"9aaa21fe0197d846207bd991c213a155b549687534e1897b36d1680356dbdb6b"} Jun 09 11:55:14.517992 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:14.517942 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-166.ec2.internal" podStartSLOduration=22.517927729 podStartE2EDuration="22.517927729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:14.499949609 +0000 UTC m=+23.781766834" watchObservedRunningTime="2026-06-09 11:55:14.517927729 +0000 UTC m=+23.799744952" Jun 09 11:55:15.196663 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:15.196631 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:15.197426 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:15.196781 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:15.204937 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:15.204861 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T11:55:14.313675359Z","UUID":"c8201653-64ac-446e-a21b-96fd553d85a2","Handler":null,"Name":"","Endpoint":""} Jun 09 11:55:15.206387 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:15.206365 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 11:55:15.206492 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:15.206405 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 11:55:16.195977 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:16.195939 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:16.196162 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:16.195944 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:16.196162 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:16.196084 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:16.196268 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:16.196161 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:16.492805 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:16.492705 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" event={"ID":"03dd62b7-4577-4874-b4c2-2cc71ae50bbd","Type":"ContainerStarted","Data":"32a546d3e671546840c880c31b2adc8c2f3342aeed87204178542e09c63a7cbc"} Jun 09 11:55:16.496454 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:16.496426 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"1e97a3f69956fa1ab592cd91cf466b6e7de13f377527bb5e7f5c8d629c6515c6"} Jun 09 11:55:16.521415 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:16.521361 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4nd6j" podStartSLOduration=5.814792684 podStartE2EDuration="25.521339861s" totalImagesPullingTime="19.706547177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:14.517619546 +0000 UTC m=+23.799436769" watchObservedRunningTime="2026-06-09 11:55:16.521339861 +0000 UTC m=+25.803157085" Jun 09 11:55:16.521657 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:16.521634 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8zj5" podStartSLOduration=3.795306164 podStartE2EDuration="25.521621145s" totalImagesPullingTime="21.726314981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:16.520933316 +0000 UTC m=+25.802750538" watchObservedRunningTime="2026-06-09 11:55:16.521621145 +0000 UTC m=+25.803438376" Jun 09 11:55:17.196006 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:17.195980 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:17.196190 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:17.196159 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:17.233211 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:17.233140 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:55:17.234403 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:17.234351 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:55:17.499126 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:17.499051 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:55:17.499790 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:17.499771 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5d8h9" Jun 09 11:55:18.195811 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.195576 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:18.195917 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.195824 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:18.195976 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:18.195909 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:18.196024 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:18.196003 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:18.503519 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.503443 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" event={"ID":"d32b83f1-94cf-4a81-9f0b-7d4afac31203","Type":"ContainerStarted","Data":"d1a2b9af473ecc65434dea66b4f78330ad3f9cad06e70b983a5a0ec93ba0da89"} Jun 09 11:55:18.503959 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.503759 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:55:18.503959 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.503823 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:55:18.505345 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.505324 2550 generic.go:356] "Generic (PLEG): container finished" podID="ea6ed8f4-01b5-484a-af18-20b31c632acd" containerID="a1be9b91e9b9430e0381bcbe3c7edc9b6f22d8cc9d5c2a75548b410bcd8ddad6" exitCode=0 Jun 09 11:55:18.505448 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.505429 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerDied","Data":"a1be9b91e9b9430e0381bcbe3c7edc9b6f22d8cc9d5c2a75548b410bcd8ddad6"} Jun 09 11:55:18.518976 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.518958 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:55:18.519149 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.519133 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:55:18.531213 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:18.531178 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" podStartSLOduration=7.271067044 podStartE2EDuration="27.531165604s" totalImagesPullingTime="20.26009856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:18.531099039 +0000 UTC m=+27.812916284" watchObservedRunningTime="2026-06-09 11:55:18.531165604 +0000 UTC m=+27.812982827" Jun 09 11:55:19.203705 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:19.203460 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:19.203705 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:19.203610 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:19.507556 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:19.507489 2550 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 09 11:55:20.131877 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.131841 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j8vvj"] Jun 09 11:55:20.132287 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.131947 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:20.132287 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:20.132059 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:20.133539 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.133517 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-knwgt"] Jun 09 11:55:20.133665 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.133612 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:20.133749 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:20.133704 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:20.145219 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.145198 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j8m4f"] Jun 09 11:55:20.145319 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.145294 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:20.145399 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:20.145377 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:20.511535 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.511444 2550 generic.go:356] "Generic (PLEG): container finished" podID="ea6ed8f4-01b5-484a-af18-20b31c632acd" containerID="66ab6795dc6fdcb47928f50e5a9a55ac0ef2d542d93281af8b036209e8619e17" exitCode=0 Jun 09 11:55:20.511996 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.511531 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerDied","Data":"66ab6795dc6fdcb47928f50e5a9a55ac0ef2d542d93281af8b036209e8619e17"} Jun 09 11:55:20.511996 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:20.511606 2550 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 09 11:55:21.515971 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:21.515937 2550 generic.go:356] "Generic (PLEG): container finished" podID="ea6ed8f4-01b5-484a-af18-20b31c632acd" containerID="fcbde72682906c1108c8dacefdb12313d80cb15bb2ebc27fec5910254fe9fc26" exitCode=0 Jun 09 11:55:21.516516 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:21.515993 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerDied","Data":"fcbde72682906c1108c8dacefdb12313d80cb15bb2ebc27fec5910254fe9fc26"} Jun 09 11:55:22.090789 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.090700 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:55:22.091025 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.091010 2550 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 09 11:55:22.111016 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.110963 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" podUID="d32b83f1-94cf-4a81-9f0b-7d4afac31203" containerName="ovnkube-controller" probeResult="failure" output="" Jun 09 11:55:22.121388 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.121351 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" podUID="d32b83f1-94cf-4a81-9f0b-7d4afac31203" containerName="ovnkube-controller" probeResult="failure" output="" Jun 09 11:55:22.196426 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.196395 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:22.196565 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.196469 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:22.196565 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:22.196495 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:22.196686 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:22.196650 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:22.196889 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:22.196768 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:22.196889 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:22.196853 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:23.992376 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:23.992337 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:23.992873 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:23.992432 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:23.992873 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:23.992520 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:55:23.992873 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:23.992564 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:55:55.992550942 +0000 UTC m=+65.274368142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:55:23.992873 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:23.992519 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:55:23.992873 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:23.992641 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret podName:d2098e65-cf15-4be3-b98c-4afe77fee72c nodeName:}" failed. No retries permitted until 2026-06-09 11:55:55.992629418 +0000 UTC m=+65.274446623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret") pod "global-pull-secret-syncer-knwgt" (UID: "d2098e65-cf15-4be3-b98c-4afe77fee72c") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:55:24.093252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:24.093215 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:24.093407 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.093354 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:55:24.093407 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.093380 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:55:24.093407 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.093392 2550 projected.go:196] Error preparing data for projected volume kube-api-access-zt8sq for pod openshift-network-diagnostics/network-check-target-j8m4f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:55:24.093523 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.093450 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq podName:223469e8-e6d4-41bc-aae4-6f4ba43ca7aa nodeName:}" failed. No retries permitted until 2026-06-09 11:55:56.093435397 +0000 UTC m=+65.375252598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zt8sq" (UniqueName: "kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq") pod "network-check-target-j8m4f" (UID: "223469e8-e6d4-41bc-aae4-6f4ba43ca7aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:55:24.196145 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:24.196116 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:24.196318 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:24.196117 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:24.196318 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.196219 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j8m4f" podUID="223469e8-e6d4-41bc-aae4-6f4ba43ca7aa" Jun 09 11:55:24.196442 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.196319 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:55:24.196442 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:24.196126 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:24.196442 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:24.196402 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-knwgt" podUID="d2098e65-cf15-4be3-b98c-4afe77fee72c" Jun 09 11:55:25.513224 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.513137 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-166.ec2.internal" event="NodeReady" Jun 09 11:55:25.513589 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.513258 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 11:55:25.587886 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.587830 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fcff6dcd4-4dqgw"] Jun 09 11:55:25.601073 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.601045 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj"] Jun 09 11:55:25.601500 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.601317 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.603658 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.603635 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 09 11:55:25.604069 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.604050 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 09 11:55:25.604553 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.604301 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-frx5g\"" Jun 09 11:55:25.604814 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.604724 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 09 11:55:25.620959 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.620937 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 09 11:55:25.621392 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.621377 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn"] Jun 09 11:55:25.621509 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.621495 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.624700 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.624683 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 09 11:55:25.625023 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.624986 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 09 11:55:25.625232 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.625218 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 09 11:55:25.625653 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.625633 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 09 11:55:25.635472 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.635454 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc"] Jun 09 11:55:25.635587 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.635572 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.637901 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.637883 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 09 11:55:25.638032 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.637924 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 09 11:55:25.638199 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.638187 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 09 11:55:25.638666 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.638651 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 09 11:55:25.649775 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.649717 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc"] Jun 09 11:55:25.649850 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.649792 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj"] Jun 09 11:55:25.649850 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.649822 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn"] Jun 09 11:55:25.649850 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.649827 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.649850 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.649835 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcff6dcd4-4dqgw"] Jun 09 11:55:25.649850 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.649850 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rhksv"] Jun 09 11:55:25.653541 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.653518 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 09 11:55:25.654046 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.653518 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tpsdr\"" Jun 09 11:55:25.671986 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.671967 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8wjnj"] Jun 09 11:55:25.672102 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.672088 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.674332 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.674313 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 11:55:25.674417 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.674361 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs666\"" Jun 09 11:55:25.674417 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.674392 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 11:55:25.686267 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.686247 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rhksv"] Jun 09 11:55:25.686362 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.686275 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8wjnj"] Jun 09 11:55:25.686362 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.686347 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.689064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.689047 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 11:55:25.689174 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.689157 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 11:55:25.689391 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.689373 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 11:55:25.689494 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.689480 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tpmhc\"" Jun 09 11:55:25.704371 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704352 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st6lh\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-kube-api-access-st6lh\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704458 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704379 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8e8e742-ab29-499a-a61b-590221d1e221-ca-trust-extracted\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704458 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704412 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-trusted-ca\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704458 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704443 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-installation-pull-secrets\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704609 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704510 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-bound-sa-token\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704609 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704545 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704609 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704572 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-registry-certificates\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.704699 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.704634 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-image-registry-private-configuration\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.803846 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.803787 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:25.805769 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.805751 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-bound-sa-token\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.805824 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.805780 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.805824 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.805796 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-registry-certificates\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.805824 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.805814 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddlhv\" (UniqueName: \"kubernetes.io/projected/7534b1a3-71fe-46df-8218-c8703fdf5f7b-kube-api-access-ddlhv\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.805944 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.805837 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-ca\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.805944 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.805861 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c82hs\" (UniqueName: \"kubernetes.io/projected/bc83f433-9d76-424b-9236-42bd78569b7c-kube-api-access-c82hs\") pod \"ingress-canary-8wjnj\" (UID: \"bc83f433-9d76-424b-9236-42bd78569b7c\") " pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.805944 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:25.805890 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:25.805944 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:25.805911 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:25.806110 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:25.805996 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:55:26.305976814 +0000 UTC m=+35.587794025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:25.806110 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806034 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7534b1a3-71fe-46df-8218-c8703fdf5f7b-tmp-dir\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.806110 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806062 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7982256d-1800-4b40-a495-40d912a5a73c-tmp\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.806110 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806080 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w76rk\" (UniqueName: \"kubernetes.io/projected/7982256d-1800-4b40-a495-40d912a5a73c-kube-api-access-w76rk\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.806110 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806100 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806118 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806145 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc83f433-9d76-424b-9236-42bd78569b7c-cert\") pod \"ingress-canary-8wjnj\" (UID: \"bc83f433-9d76-424b-9236-42bd78569b7c\") " pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806173 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlprk\" (UniqueName: \"kubernetes.io/projected/8eadd14b-f643-4296-a8fa-644d8f276771-kube-api-access-tlprk\") pod \"managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc\" (UID: \"8eadd14b-f643-4296-a8fa-644d8f276771\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806236 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-image-registry-private-configuration\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806254 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-hub\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806280 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st6lh\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-kube-api-access-st6lh\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806307 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7982256d-1800-4b40-a495-40d912a5a73c-klusterlet-config\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806338 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8e8e742-ab29-499a-a61b-590221d1e221-ca-trust-extracted\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.806365 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806362 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wh72\" (UniqueName: \"kubernetes.io/projected/b0696d5d-427c-4008-bd69-8bbd15a4eb25-kube-api-access-2wh72\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806368 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-registry-certificates\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806389 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8eadd14b-f643-4296-a8fa-644d8f276771-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc\" (UID: \"8eadd14b-f643-4296-a8fa-644d8f276771\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806417 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7534b1a3-71fe-46df-8218-c8703fdf5f7b-config-volume\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806485 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0696d5d-427c-4008-bd69-8bbd15a4eb25-additional-service-ca\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806545 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-trusted-ca\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806596 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-installation-pull-secrets\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806634 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.806777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.806658 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8e8e742-ab29-499a-a61b-590221d1e221-ca-trust-extracted\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.807305 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.807288 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-trusted-ca\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.810138 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.810119 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-installation-pull-secrets\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.810138 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.810130 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-image-registry-private-configuration\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.819249 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.819224 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-bound-sa-token\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.819799 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.819783 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st6lh\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-kube-api-access-st6lh\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:25.907100 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907071 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wh72\" (UniqueName: \"kubernetes.io/projected/b0696d5d-427c-4008-bd69-8bbd15a4eb25-kube-api-access-2wh72\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.907191 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907103 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8eadd14b-f643-4296-a8fa-644d8f276771-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc\" (UID: \"8eadd14b-f643-4296-a8fa-644d8f276771\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.907191 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907128 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7534b1a3-71fe-46df-8218-c8703fdf5f7b-config-volume\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.907191 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907178 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0696d5d-427c-4008-bd69-8bbd15a4eb25-additional-service-ca\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.907346 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907221 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.907573 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907550 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddlhv\" (UniqueName: \"kubernetes.io/projected/7534b1a3-71fe-46df-8218-c8703fdf5f7b-kube-api-access-ddlhv\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.907663 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907591 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-ca\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.907663 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907616 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c82hs\" (UniqueName: \"kubernetes.io/projected/bc83f433-9d76-424b-9236-42bd78569b7c-kube-api-access-c82hs\") pod \"ingress-canary-8wjnj\" (UID: \"bc83f433-9d76-424b-9236-42bd78569b7c\") " pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.907663 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907655 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7534b1a3-71fe-46df-8218-c8703fdf5f7b-tmp-dir\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.907908 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907678 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7982256d-1800-4b40-a495-40d912a5a73c-tmp\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.907908 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907701 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w76rk\" (UniqueName: \"kubernetes.io/projected/7982256d-1800-4b40-a495-40d912a5a73c-kube-api-access-w76rk\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.907908 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907730 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.907908 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907788 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.907908 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907807 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7534b1a3-71fe-46df-8218-c8703fdf5f7b-config-volume\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.908158 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907932 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0696d5d-427c-4008-bd69-8bbd15a4eb25-additional-service-ca\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.908158 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.908041 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7534b1a3-71fe-46df-8218-c8703fdf5f7b-tmp-dir\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.908158 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:25.908077 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:25.908158 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.907817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc83f433-9d76-424b-9236-42bd78569b7c-cert\") pod \"ingress-canary-8wjnj\" (UID: \"bc83f433-9d76-424b-9236-42bd78569b7c\") " pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.908158 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.908127 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlprk\" (UniqueName: \"kubernetes.io/projected/8eadd14b-f643-4296-a8fa-644d8f276771-kube-api-access-tlprk\") pod \"managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc\" (UID: \"8eadd14b-f643-4296-a8fa-644d8f276771\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.908396 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.908164 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7982256d-1800-4b40-a495-40d912a5a73c-tmp\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.908396 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.908185 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-hub\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.908396 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.908215 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7982256d-1800-4b40-a495-40d912a5a73c-klusterlet-config\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.908396 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:25.908335 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:55:26.408315705 +0000 UTC m=+35.690132908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:55:25.910049 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.910028 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8eadd14b-f643-4296-a8fa-644d8f276771-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc\" (UID: \"8eadd14b-f643-4296-a8fa-644d8f276771\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.910588 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.910563 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-ca\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.910676 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.910617 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.910928 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.910906 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.910982 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.910941 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc83f433-9d76-424b-9236-42bd78569b7c-cert\") pod \"ingress-canary-8wjnj\" (UID: \"bc83f433-9d76-424b-9236-42bd78569b7c\") " pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.911093 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.911074 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b0696d5d-427c-4008-bd69-8bbd15a4eb25-hub\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.911444 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.911428 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7982256d-1800-4b40-a495-40d912a5a73c-klusterlet-config\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.915576 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.915553 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wh72\" (UniqueName: \"kubernetes.io/projected/b0696d5d-427c-4008-bd69-8bbd15a4eb25-kube-api-access-2wh72\") pod \"cluster-proxy-proxy-agent-645966b9cc-bgjdn\" (UID: \"b0696d5d-427c-4008-bd69-8bbd15a4eb25\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.916999 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.916973 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:25.918428 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.918406 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddlhv\" (UniqueName: \"kubernetes.io/projected/7534b1a3-71fe-46df-8218-c8703fdf5f7b-kube-api-access-ddlhv\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:25.918528 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.918440 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w76rk\" (UniqueName: \"kubernetes.io/projected/7982256d-1800-4b40-a495-40d912a5a73c-kube-api-access-w76rk\") pod \"klusterlet-addon-workmgr-56b4d8b589-5jpjj\" (UID: \"7982256d-1800-4b40-a495-40d912a5a73c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.918663 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.918645 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c82hs\" (UniqueName: \"kubernetes.io/projected/bc83f433-9d76-424b-9236-42bd78569b7c-kube-api-access-c82hs\") pod \"ingress-canary-8wjnj\" (UID: \"bc83f433-9d76-424b-9236-42bd78569b7c\") " pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:25.918712 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.918683 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlprk\" (UniqueName: \"kubernetes.io/projected/8eadd14b-f643-4296-a8fa-644d8f276771-kube-api-access-tlprk\") pod \"managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc\" (UID: \"8eadd14b-f643-4296-a8fa-644d8f276771\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.931555 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.931535 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:25.954984 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.954968 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 11:55:25.961144 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.961120 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:55:25.995347 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:25.995322 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8wjnj" Jun 09 11:55:26.097751 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.097674 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj"] Jun 09 11:55:26.108004 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:55:26.107963 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7982256d_1800_4b40_a495_40d912a5a73c.slice/crio-fb22cf5228fba7b8d96a4cff13e3b877734b571d907d2a6949af04848e17dc74 WatchSource:0}: Error finding container fb22cf5228fba7b8d96a4cff13e3b877734b571d907d2a6949af04848e17dc74: Status 404 returned error can't find the container with id fb22cf5228fba7b8d96a4cff13e3b877734b571d907d2a6949af04848e17dc74 Jun 09 11:55:26.196175 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.196102 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:26.196289 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.196243 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:26.196641 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.196407 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:26.200481 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.200236 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 11:55:26.200481 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.200274 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 11:55:26.200872 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.200848 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 11:55:26.201493 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.201132 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 11:55:26.201493 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.201213 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mq88t\"" Jun 09 11:55:26.201493 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.201301 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wmll9\"" Jun 09 11:55:26.238953 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.238931 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn"] Jun 09 11:55:26.244586 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.244567 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc"] Jun 09 11:55:26.245366 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.245346 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8wjnj"] Jun 09 11:55:26.247503 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:55:26.247478 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8eadd14b_f643_4296_a8fa_644d8f276771.slice/crio-d9d653a6ecab9057e56dcdc54e8a2363957c1cef174cfc7722f4fbc1f64f3e8c WatchSource:0}: Error finding container d9d653a6ecab9057e56dcdc54e8a2363957c1cef174cfc7722f4fbc1f64f3e8c: Status 404 returned error can't find the container with id d9d653a6ecab9057e56dcdc54e8a2363957c1cef174cfc7722f4fbc1f64f3e8c Jun 09 11:55:26.247889 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:55:26.247847 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0696d5d_427c_4008_bd69_8bbd15a4eb25.slice/crio-133bd1ead293ab1f2fc17b9afd7e096275d5db68cae51c9ac38d6e7efbf9fd00 WatchSource:0}: Error finding container 133bd1ead293ab1f2fc17b9afd7e096275d5db68cae51c9ac38d6e7efbf9fd00: Status 404 returned error can't find the container with id 133bd1ead293ab1f2fc17b9afd7e096275d5db68cae51c9ac38d6e7efbf9fd00 Jun 09 11:55:26.248576 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:55:26.248489 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc83f433_9d76_424b_9236_42bd78569b7c.slice/crio-0da72f3b0878353a8226f320904408b8d22114766c4af048d898acf0233770d1 WatchSource:0}: Error finding container 0da72f3b0878353a8226f320904408b8d22114766c4af048d898acf0233770d1: Status 404 returned error can't find the container with id 0da72f3b0878353a8226f320904408b8d22114766c4af048d898acf0233770d1 Jun 09 11:55:26.310828 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.310803 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:26.310955 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:26.310940 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:26.310990 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:26.310961 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:26.311028 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:26.311019 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:55:27.310993252 +0000 UTC m=+36.592810461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:26.396029 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.396002 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:26.412027 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.412009 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:26.412169 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:26.412154 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:26.412234 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:26.412220 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:55:27.412200538 +0000 UTC m=+36.694017748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:55:26.496969 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.496946 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:26.533405 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.533377 2550 generic.go:356] "Generic (PLEG): container finished" podID="ea6ed8f4-01b5-484a-af18-20b31c632acd" containerID="1cc699fffc48e36fd9a7dc12bd91559e682e2c08687d3e0da3641b6fb0576824" exitCode=0 Jun 09 11:55:26.533802 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.533446 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerDied","Data":"1cc699fffc48e36fd9a7dc12bd91559e682e2c08687d3e0da3641b6fb0576824"} Jun 09 11:55:26.534644 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.534622 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8wjnj" event={"ID":"bc83f433-9d76-424b-9236-42bd78569b7c","Type":"ContainerStarted","Data":"0da72f3b0878353a8226f320904408b8d22114766c4af048d898acf0233770d1"} Jun 09 11:55:26.538400 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.538378 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" event={"ID":"8eadd14b-f643-4296-a8fa-644d8f276771","Type":"ContainerStarted","Data":"d9d653a6ecab9057e56dcdc54e8a2363957c1cef174cfc7722f4fbc1f64f3e8c"} Jun 09 11:55:26.539460 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.539438 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" event={"ID":"7982256d-1800-4b40-a495-40d912a5a73c","Type":"ContainerStarted","Data":"fb22cf5228fba7b8d96a4cff13e3b877734b571d907d2a6949af04848e17dc74"} Jun 09 11:55:26.540450 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:26.540388 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" event={"ID":"b0696d5d-427c-4008-bd69-8bbd15a4eb25","Type":"ContainerStarted","Data":"133bd1ead293ab1f2fc17b9afd7e096275d5db68cae51c9ac38d6e7efbf9fd00"} Jun 09 11:55:27.239812 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.239564 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:27.321400 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.321359 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:27.321788 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:27.321571 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:27.321788 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:27.321593 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:27.321788 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:27.321653 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:55:29.321634828 +0000 UTC m=+38.603452053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:27.323717 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.323638 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:27.422449 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.421896 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:27.422449 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:27.422081 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:27.422449 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:27.422136 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:55:29.422119586 +0000 UTC m=+38.703936787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:55:27.423978 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.423921 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:27.552127 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.552044 2550 generic.go:356] "Generic (PLEG): container finished" podID="ea6ed8f4-01b5-484a-af18-20b31c632acd" containerID="f1d9dd420fd83ad94614b29faf0438403987d7b7257d130a2b3804fc1af2d547" exitCode=0 Jun 09 11:55:27.552127 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:27.552121 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerDied","Data":"f1d9dd420fd83ad94614b29faf0438403987d7b7257d130a2b3804fc1af2d547"} Jun 09 11:55:28.565847 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:28.565595 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" event={"ID":"ea6ed8f4-01b5-484a-af18-20b31c632acd","Type":"ContainerStarted","Data":"83bd6343900ddd3c24d31eff6f90edb43581e8ad629af030842a40d3e4ff782c"} Jun 09 11:55:28.596923 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:28.596868 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7cnfm" podStartSLOduration=6.972211263 podStartE2EDuration="37.596846615s" totalImagesPullingTime="30.624635352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:28.595233659 +0000 UTC m=+37.877050884" watchObservedRunningTime="2026-06-09 11:55:28.596846615 +0000 UTC m=+37.878663840" Jun 09 11:55:28.758909 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:28.758830 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:28.890133 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:28.890069 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:29.346579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:29.346418 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:29.346728 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:29.346637 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:29.346728 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:29.346660 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:29.346728 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:29.346723 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:55:33.346704416 +0000 UTC m=+42.628521632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:29.447656 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:29.447622 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:29.447865 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:29.447848 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:29.447933 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:29.447918 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:55:33.447899155 +0000 UTC m=+42.729716359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:55:30.162619 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:30.162582 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:30.281546 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:30.281508 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:30.881891 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:30.881853 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:30.949926 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:30.949889 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:33.386246 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:33.386213 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:33.386639 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:33.386333 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:33.386639 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:33.386344 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:33.386639 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:33.386393 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:55:41.386379275 +0000 UTC m=+50.668196489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:33.487274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:33.487241 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:33.487440 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:33.487389 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:33.487479 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:33.487450 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:55:41.487433356 +0000 UTC m=+50.769250557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:55:35.586138 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.586107 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" event={"ID":"8eadd14b-f643-4296-a8fa-644d8f276771","Type":"ContainerStarted","Data":"4f51cc975672925ed231c783ea53af3e20c7bb0dc2dad8e1098f3e4b9d2cd20a"} Jun 09 11:55:35.587469 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.587442 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" event={"ID":"7982256d-1800-4b40-a495-40d912a5a73c","Type":"ContainerStarted","Data":"465d6c8db8e6658d2501c9261496946ea1b0c2de574531a2d6c3aed3d01d7056"} Jun 09 11:55:35.587621 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.587601 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:35.589469 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.589451 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:55:35.589662 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.589639 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" event={"ID":"b0696d5d-427c-4008-bd69-8bbd15a4eb25","Type":"ContainerStarted","Data":"703ef050da75a42c05040c4c39b85f7737227b36b3bdd99fdb10675de4561983"} Jun 09 11:55:35.589778 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.589673 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" event={"ID":"b0696d5d-427c-4008-bd69-8bbd15a4eb25","Type":"ContainerStarted","Data":"0f36adffa97a2a973e36f2302e66aca4fa354aab9f7e874055090068e50960f8"} Jun 09 11:55:35.589778 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.589688 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" event={"ID":"b0696d5d-427c-4008-bd69-8bbd15a4eb25","Type":"ContainerStarted","Data":"525709e7f08029f9de3adb06d7c451935ff3aaec41746654d599480cf6e1c0ae"} Jun 09 11:55:35.590988 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.590968 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8wjnj" event={"ID":"bc83f433-9d76-424b-9236-42bd78569b7c","Type":"ContainerStarted","Data":"3a2acafd3362610e3ff94e8e76015d7656eb300d007c60f12e8538d96a0a20d8"} Jun 09 11:55:35.602039 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.602002 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" podStartSLOduration=29.379702144 podStartE2EDuration="37.601990027s" totalImagesPullingTime="8.222287883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:35.600978977 +0000 UTC m=+44.882796200" watchObservedRunningTime="2026-06-09 11:55:35.601990027 +0000 UTC m=+44.883807227" Jun 09 11:55:35.615204 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.615169 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" podStartSLOduration=29.259749247 podStartE2EDuration="37.615154236s" totalImagesPullingTime="8.355404989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:35.614862793 +0000 UTC m=+44.896680029" watchObservedRunningTime="2026-06-09 11:55:35.615154236 +0000 UTC m=+44.896971460" Jun 09 11:55:35.632973 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.632939 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" podStartSLOduration=29.416732341 podStartE2EDuration="37.632928268s" totalImagesPullingTime="8.216195927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:35.631818761 +0000 UTC m=+44.913635983" watchObservedRunningTime="2026-06-09 11:55:35.632928268 +0000 UTC m=+44.914745493" Jun 09 11:55:35.647042 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:35.647012 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8wjnj" podStartSLOduration=2.462970303 podStartE2EDuration="10.647004498s" totalImagesPullingTime="8.184034195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:55:35.6466307 +0000 UTC m=+44.928447923" watchObservedRunningTime="2026-06-09 11:55:35.647004498 +0000 UTC m=+44.928821721" Jun 09 11:55:37.824357 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:37.824324 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:37.925122 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:37.925089 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:38.555637 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:38.555606 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:38.655375 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:38.655343 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:38.788745 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:38.788694 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:38.889402 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:38.889374 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:39.529626 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:39.529597 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:41.443070 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:41.443035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:41.443441 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:41.443202 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:41.443441 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:41.443222 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:41.443441 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:41.443293 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:55:57.443273734 +0000 UTC m=+66.725090935 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:41.544237 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:41.544210 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:41.544343 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:41.544330 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:41.544394 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:41.544385 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:55:57.544372504 +0000 UTC m=+66.826189705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:55:42.367635 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:42.367601 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:42.465507 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:42.465476 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:43.754811 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:43.754778 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:43.806684 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:43.806655 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:51.109457 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:51.109386 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:52.120511 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:52.120481 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7zd" Jun 09 11:55:52.287254 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:52.287219 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:52.385843 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:52.385812 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:53.863416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:53.863383 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:53.964783 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:53.964751 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:55.606373 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:55.606337 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:55.660291 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:55.660263 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:55:56.061445 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.061411 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:55:56.061621 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.061455 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:56.064006 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.063988 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 11:55:56.064090 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.064002 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 11:55:56.072049 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:56.072033 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 11:55:56.072111 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:56.072099 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:57:00.0720788 +0000 UTC m=+129.353896003 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : secret "metrics-daemon-secret" not found Jun 09 11:55:56.075181 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.075152 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2098e65-cf15-4be3-b98c-4afe77fee72c-original-pull-secret\") pod \"global-pull-secret-syncer-knwgt\" (UID: \"d2098e65-cf15-4be3-b98c-4afe77fee72c\") " pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:56.162728 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.162697 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:56.165621 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.165601 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 11:55:56.175504 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.175486 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 11:55:56.186749 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.186718 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt8sq\" (UniqueName: \"kubernetes.io/projected/223469e8-e6d4-41bc-aae4-6f4ba43ca7aa-kube-api-access-zt8sq\") pod \"network-check-target-j8m4f\" (UID: \"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa\") " pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:56.214485 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.214458 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-knwgt" Jun 09 11:55:56.223029 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.223007 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mq88t\"" Jun 09 11:55:56.230976 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.230954 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:55:56.374866 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.374838 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-knwgt"] Jun 09 11:55:56.378071 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:55:56.378044 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2098e65_cf15_4be3_b98c_4afe77fee72c.slice/crio-4d38d4f0b3fca34b8417a1e53eef051e01422f90fd547840b7ae84d1801bd1b2 WatchSource:0}: Error finding container 4d38d4f0b3fca34b8417a1e53eef051e01422f90fd547840b7ae84d1801bd1b2: Status 404 returned error can't find the container with id 4d38d4f0b3fca34b8417a1e53eef051e01422f90fd547840b7ae84d1801bd1b2 Jun 09 11:55:56.394535 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.394479 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j8m4f"] Jun 09 11:55:56.396676 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:55:56.396655 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223469e8_e6d4_41bc_aae4_6f4ba43ca7aa.slice/crio-1e057dd471161897363ed52cf46d1de788722abfc8be25e2fa34b64ac607d7ac WatchSource:0}: Error finding container 1e057dd471161897363ed52cf46d1de788722abfc8be25e2fa34b64ac607d7ac: Status 404 returned error can't find the container with id 1e057dd471161897363ed52cf46d1de788722abfc8be25e2fa34b64ac607d7ac Jun 09 11:55:56.648598 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.648561 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j8m4f" event={"ID":"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa","Type":"ContainerStarted","Data":"1e057dd471161897363ed52cf46d1de788722abfc8be25e2fa34b64ac607d7ac"} Jun 09 11:55:56.649654 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:56.649633 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-knwgt" event={"ID":"d2098e65-cf15-4be3-b98c-4afe77fee72c","Type":"ContainerStarted","Data":"4d38d4f0b3fca34b8417a1e53eef051e01422f90fd547840b7ae84d1801bd1b2"} Jun 09 11:55:57.471794 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:57.471190 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:55:57.471794 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:57.471402 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:55:57.471794 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:57.471420 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:55:57.471794 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:57.471481 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:56:29.471461259 +0000 UTC m=+98.753278466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:55:57.572184 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:55:57.572119 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:55:57.572367 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:57.572323 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:55:57.572438 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:55:57.572431 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:56:29.572409089 +0000 UTC m=+98.854226307 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:56:01.668877 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:01.668835 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-knwgt" event={"ID":"d2098e65-cf15-4be3-b98c-4afe77fee72c","Type":"ContainerStarted","Data":"3e900c97eac5f234b5ee268afc0f0d95ec4327be3bda9914e9e9ab00b8f17985"} Jun 09 11:56:01.670251 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:01.670191 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j8m4f" event={"ID":"223469e8-e6d4-41bc-aae4-6f4ba43ca7aa","Type":"ContainerStarted","Data":"b26be99f184bdbe7876a6387ab5784208cb57fa9799650361f61a1f54c4712e9"} Jun 09 11:56:01.670339 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:01.670322 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:56:01.685959 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:01.685919 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-knwgt" podStartSLOduration=66.341153096 podStartE2EDuration="1m10.685903328s" totalImagesPullingTime="4.344750232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:56:01.685042447 +0000 UTC m=+70.966859672" watchObservedRunningTime="2026-06-09 11:56:01.685903328 +0000 UTC m=+70.967720552" Jun 09 11:56:01.699434 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:01.699397 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j8m4f" podStartSLOduration=66.378553595 podStartE2EDuration="1m10.699385999s" totalImagesPullingTime="4.320832404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:56:01.699094221 +0000 UTC m=+70.980911443" watchObservedRunningTime="2026-06-09 11:56:01.699385999 +0000 UTC m=+70.981203221" Jun 09 11:56:06.448073 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:06.448044 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:56:06.500883 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:06.500843 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:56:15.845681 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:15.845569 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:56:15.951932 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:15.951901 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:56:29.495547 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:29.495505 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:56:29.495996 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:56:29.495633 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:56:29.495996 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:56:29.495652 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:56:29.495996 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:56:29.495723 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:57:33.49570149 +0000 UTC m=+162.777518711 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:56:29.596119 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:29.596087 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:56:29.596230 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:56:29.596210 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:56:29.596266 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:56:29.596262 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:57:33.596250047 +0000 UTC m=+162.878067248 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:56:32.675622 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:32.675594 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j8m4f" Jun 09 11:56:34.060274 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:56:34.060240 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:57:00.114952 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:00.114909 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:57:00.115440 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:00.115030 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 11:57:00.115440 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:00.115083 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 11:59:02.115070085 +0000 UTC m=+251.396887286 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : secret "metrics-daemon-secret" not found Jun 09 11:57:28.613808 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:28.613771 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" Jun 09 11:57:28.690463 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:28.690428 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rhksv" podUID="7534b1a3-71fe-46df-8218-c8703fdf5f7b" Jun 09 11:57:28.922476 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:28.922446 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rhksv" Jun 09 11:57:28.922476 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:28.922483 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:57:29.208539 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:29.208455 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:57:33.546883 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:33.546839 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:57:33.547279 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:33.546990 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 11:57:33.547279 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:33.547008 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcff6dcd4-4dqgw: secret "image-registry-tls" not found Jun 09 11:57:33.547279 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:33.547080 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls podName:e8e8e742-ab29-499a-a61b-590221d1e221 nodeName:}" failed. No retries permitted until 2026-06-09 11:59:35.547063008 +0000 UTC m=+284.828880209 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls") pod "image-registry-7fcff6dcd4-4dqgw" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221") : secret "image-registry-tls" not found Jun 09 11:57:33.648199 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:33.648167 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:57:33.648320 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:33.648284 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 11:57:33.648356 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:57:33.648345 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls podName:7534b1a3-71fe-46df-8218-c8703fdf5f7b nodeName:}" failed. No retries permitted until 2026-06-09 11:59:35.648333094 +0000 UTC m=+284.930150295 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls") pod "dns-default-rhksv" (UID: "7534b1a3-71fe-46df-8218-c8703fdf5f7b") : secret "dns-default-metrics-tls" not found Jun 09 11:57:34.938294 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:34.938259 2550 generic.go:356] "Generic (PLEG): container finished" podID="7982256d-1800-4b40-a495-40d912a5a73c" containerID="465d6c8db8e6658d2501c9261496946ea1b0c2de574531a2d6c3aed3d01d7056" exitCode=1 Jun 09 11:57:34.938772 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:34.938337 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" event={"ID":"7982256d-1800-4b40-a495-40d912a5a73c","Type":"ContainerDied","Data":"465d6c8db8e6658d2501c9261496946ea1b0c2de574531a2d6c3aed3d01d7056"} Jun 09 11:57:34.938772 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:34.938649 2550 scope.go:117] "RemoveContainer" containerID="465d6c8db8e6658d2501c9261496946ea1b0c2de574531a2d6c3aed3d01d7056" Jun 09 11:57:34.939988 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:34.939875 2550 generic.go:356] "Generic (PLEG): container finished" podID="8eadd14b-f643-4296-a8fa-644d8f276771" containerID="4f51cc975672925ed231c783ea53af3e20c7bb0dc2dad8e1098f3e4b9d2cd20a" exitCode=255 Jun 09 11:57:34.939988 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:34.939911 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" event={"ID":"8eadd14b-f643-4296-a8fa-644d8f276771","Type":"ContainerDied","Data":"4f51cc975672925ed231c783ea53af3e20c7bb0dc2dad8e1098f3e4b9d2cd20a"} Jun 09 11:57:34.940180 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:34.940164 2550 scope.go:117] "RemoveContainer" containerID="4f51cc975672925ed231c783ea53af3e20c7bb0dc2dad8e1098f3e4b9d2cd20a" Jun 09 11:57:35.588580 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:35.588534 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:57:35.931808 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:35.931774 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:57:35.944370 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:35.944341 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" event={"ID":"8eadd14b-f643-4296-a8fa-644d8f276771","Type":"ContainerStarted","Data":"aaf60859bf80a4b2c54ecd8c08b953017b30a867ff1a13955b5b9aec677b0b65"} Jun 09 11:57:35.946041 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:35.946022 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" event={"ID":"7982256d-1800-4b40-a495-40d912a5a73c","Type":"ContainerStarted","Data":"777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b"} Jun 09 11:57:35.946235 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:35.946219 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:57:35.946777 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:35.946761 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:57:43.196558 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:57:43.196467 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:58:08.927456 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:58:08.927418 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:02.142452 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:02.142400 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:59:02.143058 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:02.142547 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 11:59:02.143058 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:02.142622 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs podName:de637c06-0362-4c2d-b07f-af51cc52e6ce nodeName:}" failed. No retries permitted until 2026-06-09 12:01:04.142604856 +0000 UTC m=+373.424422057 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs") pod "network-metrics-daemon-j8vvj" (UID: "de637c06-0362-4c2d-b07f-af51cc52e6ce") : secret "metrics-daemon-secret" not found Jun 09 11:59:03.824605 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:03.824566 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:03.892165 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:03.892133 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:08.874398 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:08.874363 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:08.974752 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:08.974703 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:11.336084 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:11.336007 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:12.494568 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:12.494536 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:12.578590 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:12.578558 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:12.635042 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:12.635006 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:13.067887 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:13.067854 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:13.179111 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:13.179082 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:13.296426 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:13.296400 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:14.329307 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:14.329283 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p9fst_f55d0bdb-53c4-4e01-b6e3-be9388a0de06/dns-node-resolver/0.log" Jun 09 11:59:14.829560 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:14.829527 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:14.895017 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:14.894991 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:15.343918 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:15.343890 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:15.532887 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:15.532863 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kzjmq_0d0a2e7e-9116-4e14-a83e-2708fd94c97b/node-ca/0.log" Jun 09 11:59:15.776684 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:15.776659 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:16.329331 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:16.329303 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8wjnj_bc83f433-9d76-424b-9236-42bd78569b7c/serve-healthcheck-canary/0.log" Jun 09 11:59:16.605835 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:16.605752 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:16.844291 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:16.844239 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:16.939001 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:16.938975 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:17.041536 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:17.041509 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:17.312730 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:17.312654 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:18.485190 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:18.485153 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:18.682564 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:18.682535 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:18.878291 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:18.878233 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:19.080093 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:19.080051 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:19.272841 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:19.272725 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:20.278709 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:20.278678 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:20.378478 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:20.378453 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:20.479640 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:20.479614 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:20.579094 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:20.579021 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:20.782148 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:20.782117 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:20.908165 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:20.908136 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:21.070306 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:21.070274 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:21.475771 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:21.475724 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:22.271800 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:22.271763 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:22.669747 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:22.669713 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:22.880630 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:22.880587 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:23.075608 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:23.075529 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:23.418996 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:23.418969 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:23.519579 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:23.519550 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:24.077638 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:24.077607 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:24.477976 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:24.477947 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:24.880117 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:24.880087 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:25.074110 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:25.074084 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:25.493849 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:25.493819 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:25.672243 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:25.672211 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:25.875713 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:25.875678 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:26.290990 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:26.290888 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:26.911610 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:26.911574 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:26.996663 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:26.996618 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:27.096917 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:27.096888 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:27.198055 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:27.197975 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:27.297710 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:27.297681 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:27.676148 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:27.676111 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:27.730868 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:27.730823 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:29.424040 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:29.424010 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:29.480726 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:29.480689 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:31.923013 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:31.922973 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" Jun 09 11:59:31.923013 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:31.922973 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rhksv" podUID="7534b1a3-71fe-46df-8218-c8703fdf5f7b" Jun 09 11:59:32.266754 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:32.266669 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:59:32.266919 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:32.266674 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rhksv" Jun 09 11:59:35.275890 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.275862 2550 generic.go:356] "Generic (PLEG): container finished" podID="8eadd14b-f643-4296-a8fa-644d8f276771" containerID="aaf60859bf80a4b2c54ecd8c08b953017b30a867ff1a13955b5b9aec677b0b65" exitCode=255 Jun 09 11:59:35.276371 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.275937 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" event={"ID":"8eadd14b-f643-4296-a8fa-644d8f276771","Type":"ContainerDied","Data":"aaf60859bf80a4b2c54ecd8c08b953017b30a867ff1a13955b5b9aec677b0b65"} Jun 09 11:59:35.276371 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.275988 2550 scope.go:117] "RemoveContainer" containerID="4f51cc975672925ed231c783ea53af3e20c7bb0dc2dad8e1098f3e4b9d2cd20a" Jun 09 11:59:35.276371 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.276238 2550 scope.go:117] "RemoveContainer" containerID="aaf60859bf80a4b2c54ecd8c08b953017b30a867ff1a13955b5b9aec677b0b65" Jun 09 11:59:35.276509 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:35.276416 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc_open-cluster-management-agent-addon(8eadd14b-f643-4296-a8fa-644d8f276771)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" podUID="8eadd14b-f643-4296-a8fa-644d8f276771" Jun 09 11:59:35.277675 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.277655 2550 generic.go:356] "Generic (PLEG): container finished" podID="7982256d-1800-4b40-a495-40d912a5a73c" containerID="777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b" exitCode=1 Jun 09 11:59:35.277762 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.277687 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" event={"ID":"7982256d-1800-4b40-a495-40d912a5a73c","Type":"ContainerDied","Data":"777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b"} Jun 09 11:59:35.277977 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.277963 2550 scope.go:117] "RemoveContainer" containerID="777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b" Jun 09 11:59:35.278111 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:35.278095 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-56b4d8b589-5jpjj_open-cluster-management-agent-addon(7982256d-1800-4b40-a495-40d912a5a73c)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" podUID="7982256d-1800-4b40-a495-40d912a5a73c" Jun 09 11:59:35.287083 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.287064 2550 scope.go:117] "RemoveContainer" containerID="465d6c8db8e6658d2501c9261496946ea1b0c2de574531a2d6c3aed3d01d7056" Jun 09 11:59:35.584322 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.584220 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:59:35.586485 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.586466 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"image-registry-7fcff6dcd4-4dqgw\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:59:35.684810 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.684778 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:59:35.686762 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.686747 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7534b1a3-71fe-46df-8218-c8703fdf5f7b-metrics-tls\") pod \"dns-default-rhksv\" (UID: \"7534b1a3-71fe-46df-8218-c8703fdf5f7b\") " pod="openshift-dns/dns-default-rhksv" Jun 09 11:59:35.871778 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.871705 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xs666\"" Jun 09 11:59:35.871778 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.871717 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-frx5g\"" Jun 09 11:59:35.878863 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.878844 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rhksv" Jun 09 11:59:35.878916 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.878870 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:59:35.932649 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.932543 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:59:35.946863 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.946833 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:59:35.962416 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:35.962285 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" Jun 09 11:59:36.009890 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.009856 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rhksv"] Jun 09 11:59:36.014036 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:59:36.014008 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7534b1a3_71fe_46df_8218_c8703fdf5f7b.slice/crio-6ae4fd270633cca30187ffc0da031c24e8f6e718d4fd50b89a7082699d8f0277 WatchSource:0}: Error finding container 6ae4fd270633cca30187ffc0da031c24e8f6e718d4fd50b89a7082699d8f0277: Status 404 returned error can't find the container with id 6ae4fd270633cca30187ffc0da031c24e8f6e718d4fd50b89a7082699d8f0277 Jun 09 11:59:36.027904 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.027882 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcff6dcd4-4dqgw"] Jun 09 11:59:36.030624 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:59:36.030606 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e8e742_ab29_499a_a61b_590221d1e221.slice/crio-531d154b7cd49126a8322f2173e8711c82bef8e5ed679ead7dc659e4b74bf7c4 WatchSource:0}: Error finding container 531d154b7cd49126a8322f2173e8711c82bef8e5ed679ead7dc659e4b74bf7c4: Status 404 returned error can't find the container with id 531d154b7cd49126a8322f2173e8711c82bef8e5ed679ead7dc659e4b74bf7c4 Jun 09 11:59:36.282200 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.282160 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rhksv" event={"ID":"7534b1a3-71fe-46df-8218-c8703fdf5f7b","Type":"ContainerStarted","Data":"6ae4fd270633cca30187ffc0da031c24e8f6e718d4fd50b89a7082699d8f0277"} Jun 09 11:59:36.283604 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.283573 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" event={"ID":"e8e8e742-ab29-499a-a61b-590221d1e221","Type":"ContainerStarted","Data":"09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72"} Jun 09 11:59:36.283746 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.283620 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" event={"ID":"e8e8e742-ab29-499a-a61b-590221d1e221","Type":"ContainerStarted","Data":"531d154b7cd49126a8322f2173e8711c82bef8e5ed679ead7dc659e4b74bf7c4"} Jun 09 11:59:36.283746 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.283660 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:59:36.285292 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.285274 2550 scope.go:117] "RemoveContainer" containerID="aaf60859bf80a4b2c54ecd8c08b953017b30a867ff1a13955b5b9aec677b0b65" Jun 09 11:59:36.285447 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:36.285426 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc_open-cluster-management-agent-addon(8eadd14b-f643-4296-a8fa-644d8f276771)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" podUID="8eadd14b-f643-4296-a8fa-644d8f276771" Jun 09 11:59:36.286695 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.286678 2550 scope.go:117] "RemoveContainer" containerID="777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b" Jun 09 11:59:36.286834 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:36.286817 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-56b4d8b589-5jpjj_open-cluster-management-agent-addon(7982256d-1800-4b40-a495-40d912a5a73c)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" podUID="7982256d-1800-4b40-a495-40d912a5a73c" Jun 09 11:59:36.304578 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:36.304533 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" podStartSLOduration=285.304519934 podStartE2EDuration="4m45.304519934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:59:36.303974222 +0000 UTC m=+285.585791445" watchObservedRunningTime="2026-06-09 11:59:36.304519934 +0000 UTC m=+285.586337162" Jun 09 11:59:37.290631 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:37.290597 2550 scope.go:117] "RemoveContainer" containerID="777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b" Jun 09 11:59:37.291046 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:37.290778 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-56b4d8b589-5jpjj_open-cluster-management-agent-addon(7982256d-1800-4b40-a495-40d912a5a73c)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" podUID="7982256d-1800-4b40-a495-40d912a5a73c" Jun 09 11:59:38.052508 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.052480 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z5k4g"] Jun 09 11:59:38.055466 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.055448 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.057826 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.057804 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 11:59:38.057924 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.057863 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 11:59:38.058685 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.058671 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s4x4b\"" Jun 09 11:59:38.058685 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.058679 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 11:59:38.058806 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.058790 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 11:59:38.068499 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.068479 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z5k4g"] Jun 09 11:59:38.164303 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.164277 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:38.208139 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.208117 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56be4bd5-c956-4324-8b8e-d661773b4c84-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.208243 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.208147 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlq9x\" (UniqueName: \"kubernetes.io/projected/56be4bd5-c956-4324-8b8e-d661773b4c84-kube-api-access-jlq9x\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.208243 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.208172 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56be4bd5-c956-4324-8b8e-d661773b4c84-crio-socket\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.208243 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.208190 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56be4bd5-c956-4324-8b8e-d661773b4c84-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.208362 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.208262 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56be4bd5-c956-4324-8b8e-d661773b4c84-data-volume\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.263240 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.263213 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:38.295215 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.295190 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rhksv" event={"ID":"7534b1a3-71fe-46df-8218-c8703fdf5f7b","Type":"ContainerStarted","Data":"8a17ee2a00fb768da895916acf4765e4e27e7ce68ee3ddb44d46e23d80099850"} Jun 09 11:59:38.295513 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.295233 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rhksv" event={"ID":"7534b1a3-71fe-46df-8218-c8703fdf5f7b","Type":"ContainerStarted","Data":"275204372d8b9d2af0a362fa319077350b47d78507a07cdcdd4f1b47636913db"} Jun 09 11:59:38.295513 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.295269 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rhksv" Jun 09 11:59:38.309231 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309180 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56be4bd5-c956-4324-8b8e-d661773b4c84-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309231 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309210 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlq9x\" (UniqueName: \"kubernetes.io/projected/56be4bd5-c956-4324-8b8e-d661773b4c84-kube-api-access-jlq9x\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309350 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309236 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56be4bd5-c956-4324-8b8e-d661773b4c84-crio-socket\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309350 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309257 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56be4bd5-c956-4324-8b8e-d661773b4c84-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309350 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309280 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56be4bd5-c956-4324-8b8e-d661773b4c84-data-volume\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309350 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309323 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56be4bd5-c956-4324-8b8e-d661773b4c84-crio-socket\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309572 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309555 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56be4bd5-c956-4324-8b8e-d661773b4c84-data-volume\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.309862 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.309843 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56be4bd5-c956-4324-8b8e-d661773b4c84-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.311531 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.311511 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56be4bd5-c956-4324-8b8e-d661773b4c84-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.316859 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.316824 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rhksv" podStartSLOduration=251.919389647 podStartE2EDuration="4m13.316810422s" totalImagesPullingTime="1.397420775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:59:38.315347223 +0000 UTC m=+287.597164448" watchObservedRunningTime="2026-06-09 11:59:38.316810422 +0000 UTC m=+287.598627644" Jun 09 11:59:38.324419 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.324402 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlq9x\" (UniqueName: \"kubernetes.io/projected/56be4bd5-c956-4324-8b8e-d661773b4c84-kube-api-access-jlq9x\") pod \"insights-runtime-extractor-z5k4g\" (UID: \"56be4bd5-c956-4324-8b8e-d661773b4c84\") " pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.363052 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.363030 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:38.363927 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.363913 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z5k4g" Jun 09 11:59:38.470458 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.470422 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:38.484558 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.484533 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z5k4g"] Jun 09 11:59:38.487390 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:59:38.487351 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56be4bd5_c956_4324_8b8e_d661773b4c84.slice/crio-e3cfa10be83f926c18fe010804a8511a62015efc8150b0bef775444a6b9496a6 WatchSource:0}: Error finding container e3cfa10be83f926c18fe010804a8511a62015efc8150b0bef775444a6b9496a6: Status 404 returned error can't find the container with id e3cfa10be83f926c18fe010804a8511a62015efc8150b0bef775444a6b9496a6 Jun 09 11:59:38.538204 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:38.538178 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:39.299049 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:39.298967 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5k4g" event={"ID":"56be4bd5-c956-4324-8b8e-d661773b4c84","Type":"ContainerStarted","Data":"d5403e57f2b9a976776bad7b56ee95f8dcb558c233aa8b2eee31ee6706baec43"} Jun 09 11:59:39.299049 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:39.299028 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5k4g" event={"ID":"56be4bd5-c956-4324-8b8e-d661773b4c84","Type":"ContainerStarted","Data":"89482919ac2a01c8ec8e560a66043aab4ba07469d11497edad747a3e8ab04753"} Jun 09 11:59:39.299425 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:39.299057 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5k4g" event={"ID":"56be4bd5-c956-4324-8b8e-d661773b4c84","Type":"ContainerStarted","Data":"e3cfa10be83f926c18fe010804a8511a62015efc8150b0bef775444a6b9496a6"} Jun 09 11:59:41.175391 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:41.175351 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:41.254494 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:41.254462 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:41.308217 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:41.308187 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z5k4g" event={"ID":"56be4bd5-c956-4324-8b8e-d661773b4c84","Type":"ContainerStarted","Data":"5dae2727beec90a9975e386f8d60469336003282c268925a04ee0a8662ec08f0"} Jun 09 11:59:41.328086 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:41.328036 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z5k4g" podStartSLOduration=1.557210443 podStartE2EDuration="3.328022786s" totalImagesPullingTime="1.770812343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:59:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:59:41.327244688 +0000 UTC m=+290.609061910" watchObservedRunningTime="2026-06-09 11:59:41.328022786 +0000 UTC m=+290.609840008" Jun 09 11:59:41.397627 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:41.397589 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:43.176628 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:43.176594 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:45.321458 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.321428 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:45.544107 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.544073 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:45.594912 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.594844 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-46r7b"] Jun 09 11:59:45.599311 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.599295 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.602352 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.602324 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 11:59:45.602459 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.602411 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5k8wf\"" Jun 09 11:59:45.602539 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.602497 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 11:59:45.602697 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.602539 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 11:59:45.603131 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.603115 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 11:59:45.603206 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.603159 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 11:59:45.603206 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.603180 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 11:59:45.643543 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.643520 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:45.743487 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.743454 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:45.762558 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762536 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-sys\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762614 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762567 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762614 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762591 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-accelerators-collector-config\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762728 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762713 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762782 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762755 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59c27d58-f420-4474-9cd7-fb9c12b2046b-metrics-client-ca\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762782 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762774 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfn47\" (UniqueName: \"kubernetes.io/projected/59c27d58-f420-4474-9cd7-fb9c12b2046b-kube-api-access-vfn47\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762846 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762792 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-root\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762846 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762807 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-textfile\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.762846 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.762823 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-wtmp\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.845928 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.845862 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:45.863976 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.863949 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864097 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.863984 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59c27d58-f420-4474-9cd7-fb9c12b2046b-metrics-client-ca\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864097 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864007 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfn47\" (UniqueName: \"kubernetes.io/projected/59c27d58-f420-4474-9cd7-fb9c12b2046b-kube-api-access-vfn47\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864097 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:45.864091 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 09 11:59:45.864252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864117 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-root\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864252 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:45.864154 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls podName:59c27d58-f420-4474-9cd7-fb9c12b2046b nodeName:}" failed. No retries permitted until 2026-06-09 11:59:46.364139623 +0000 UTC m=+295.645956824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls") pod "node-exporter-46r7b" (UID: "59c27d58-f420-4474-9cd7-fb9c12b2046b") : secret "node-exporter-tls" not found Jun 09 11:59:45.864252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864171 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-root\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-textfile\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864197 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-wtmp\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864252 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864242 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-sys\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864544 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864260 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864544 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864316 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-accelerators-collector-config\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864544 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864324 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-sys\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864544 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864401 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-wtmp\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864544 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864458 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-textfile\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864544 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864493 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59c27d58-f420-4474-9cd7-fb9c12b2046b-metrics-client-ca\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.864890 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.864845 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-accelerators-collector-config\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.866502 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.866480 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.874347 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.874322 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfn47\" (UniqueName: \"kubernetes.io/projected/59c27d58-f420-4474-9cd7-fb9c12b2046b-kube-api-access-vfn47\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:45.977418 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:45.977390 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:46.119045 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.118950 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:46.197894 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:46.197863 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-j8vvj" podUID="de637c06-0362-4c2d-b07f-af51cc52e6ce" Jun 09 11:59:46.256968 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.256945 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:46.369759 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.369674 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:46.370058 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:46.369813 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 09 11:59:46.370058 ip-10-0-139-166 kubenswrapper[2550]: E0609 11:59:46.369868 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls podName:59c27d58-f420-4474-9cd7-fb9c12b2046b nodeName:}" failed. No retries permitted until 2026-06-09 11:59:47.369851801 +0000 UTC m=+296.651669022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls") pod "node-exporter-46r7b" (UID: "59c27d58-f420-4474-9cd7-fb9c12b2046b") : secret "node-exporter-tls" not found Jun 09 11:59:46.558095 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.558063 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:46.649419 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.649383 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:46.771180 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.771121 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:46.875452 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:46.875397 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:47.377496 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:47.377466 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:47.379651 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:47.379630 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/59c27d58-f420-4474-9cd7-fb9c12b2046b-node-exporter-tls\") pod \"node-exporter-46r7b\" (UID: \"59c27d58-f420-4474-9cd7-fb9c12b2046b\") " pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:47.408723 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:47.408700 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-46r7b" Jun 09 11:59:47.419780 ip-10-0-139-166 kubenswrapper[2550]: W0609 11:59:47.419760 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c27d58_f420_4474_9cd7_fb9c12b2046b.slice/crio-5fb2b73ef7f4eb55669f275b458c4385073e4d2ba56571c75e4c3fe05c074377 WatchSource:0}: Error finding container 5fb2b73ef7f4eb55669f275b458c4385073e4d2ba56571c75e4c3fe05c074377: Status 404 returned error can't find the container with id 5fb2b73ef7f4eb55669f275b458c4385073e4d2ba56571c75e4c3fe05c074377 Jun 09 11:59:48.196064 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.196035 2550 scope.go:117] "RemoveContainer" containerID="aaf60859bf80a4b2c54ecd8c08b953017b30a867ff1a13955b5b9aec677b0b65" Jun 09 11:59:48.301191 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.301169 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rhksv" Jun 09 11:59:48.331630 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.331592 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dcc8c44c8-76jrc" event={"ID":"8eadd14b-f643-4296-a8fa-644d8f276771","Type":"ContainerStarted","Data":"51f395e997f05950df00c9ee6c1582bd38fdac8b10e704d7dd19ce0f67cb797c"} Jun 09 11:59:48.333277 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.333248 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46r7b" event={"ID":"59c27d58-f420-4474-9cd7-fb9c12b2046b","Type":"ContainerStarted","Data":"5fb2b73ef7f4eb55669f275b458c4385073e4d2ba56571c75e4c3fe05c074377"} Jun 09 11:59:48.541363 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.541290 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:48.641539 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.641507 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:48.721669 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:48.721640 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:49.337209 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:49.337173 2550 generic.go:356] "Generic (PLEG): container finished" podID="59c27d58-f420-4474-9cd7-fb9c12b2046b" containerID="404b8ac89f5b550d90bc0eb0de0d9ed631b7eb75446cb5b9c85615e343837fd5" exitCode=0 Jun 09 11:59:49.337360 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:49.337228 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46r7b" event={"ID":"59c27d58-f420-4474-9cd7-fb9c12b2046b","Type":"ContainerDied","Data":"404b8ac89f5b550d90bc0eb0de0d9ed631b7eb75446cb5b9c85615e343837fd5"} Jun 09 11:59:49.390696 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:49.390666 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:50.196821 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.196794 2550 scope.go:117] "RemoveContainer" containerID="777b47376590ac26b1eea601e3d6f2c03548a17ec1c877fef2aaa5f18b54b66b" Jun 09 11:59:50.343678 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.343641 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46r7b" event={"ID":"59c27d58-f420-4474-9cd7-fb9c12b2046b","Type":"ContainerStarted","Data":"0a8833cd12a683cb88fc14574296b4fce62b543307928b3c975ba67a4623993e"} Jun 09 11:59:50.343855 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.343713 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46r7b" event={"ID":"59c27d58-f420-4474-9cd7-fb9c12b2046b","Type":"ContainerStarted","Data":"45ea40717fdac9a7c584cc8563a1b42d37632259dedb4580f3501e3908decca4"} Jun 09 11:59:50.345381 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.345359 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" event={"ID":"7982256d-1800-4b40-a495-40d912a5a73c","Type":"ContainerStarted","Data":"38ac47bf840f45ec240683475e525ce4fd5fd2d8fed219038d1864342c82c79a"} Jun 09 11:59:50.345634 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.345619 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:59:50.346722 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.346704 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56b4d8b589-5jpjj" Jun 09 11:59:50.367718 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.367695 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:50.374350 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.374313 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-46r7b" podStartSLOduration=4.431459974 podStartE2EDuration="5.374301295s" totalImagesPullingTime="875.466071ms" totalInitContainerRuntime="67.37525ms" isStatefulPod=true podCreationTimestamp="2026-06-09 11:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:59:50.372545032 +0000 UTC m=+299.654362254" watchObservedRunningTime="2026-06-09 11:59:50.374301295 +0000 UTC m=+299.656118567" Jun 09 11:59:50.566350 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:50.566264 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:51.155029 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:51.155004 2550 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 11:59:51.789870 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:51.789836 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:55.883236 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:55.883204 2550 patch_prober.go:28] interesting pod/image-registry-7fcff6dcd4-4dqgw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 11:59:55.883599 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:55.883254 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 11:59:56.115621 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:56.115586 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:57.196491 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.196455 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 11:59:57.272639 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.272607 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:57.294225 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.294204 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 11:59:57.426662 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.426629 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:57.525645 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.525576 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:57.626510 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.626483 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 11:59:57.679417 ip-10-0-139-166 kubenswrapper[2550]: I0609 11:59:57.679391 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:00.386137 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:00.386100 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fcff6dcd4-4dqgw"] Jun 09 12:00:00.467406 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:00.467378 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:00.525962 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:00.525938 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:00.590565 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:00.590540 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:07.425093 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:07.425061 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:07.485983 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:07.485949 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:07.552683 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:07.552650 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:07.715458 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:07.715362 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:07.886076 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:07.886012 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:08.085921 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:08.085846 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:08.288907 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:08.288865 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:08.389987 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:08.389951 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:09.298172 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:09.298121 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:09.500256 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:09.500182 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:09.590821 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:09.590717 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:10.136968 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:10.136925 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:10.236784 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:10.236751 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:10.338019 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:10.337988 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:10.403129 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:10.403061 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:15.879428 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:15.879392 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:15.978010 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:15.977977 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:19.134586 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:19.134557 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:19.202302 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:19.202266 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:22.576966 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:22.576933 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:22.629368 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:22.629299 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:23.295100 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:23.295052 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:23.396657 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:23.396619 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:23.479886 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:23.479851 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:23.543620 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:23.543578 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:25.409565 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.409525 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" containerName="registry" containerID="cri-o://09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72" gracePeriod=30 Jun 09 12:00:25.656162 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.656142 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 12:00:25.763230 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763142 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8e8e742-ab29-499a-a61b-590221d1e221-ca-trust-extracted\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763230 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763189 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-image-registry-private-configuration\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763230 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763214 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-registry-certificates\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763492 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763262 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-trusted-ca\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763492 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763287 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763492 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763320 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-installation-pull-secrets\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763492 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763344 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st6lh\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-kube-api-access-st6lh\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763492 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763377 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-bound-sa-token\") pod \"e8e8e742-ab29-499a-a61b-590221d1e221\" (UID: \"e8e8e742-ab29-499a-a61b-590221d1e221\") " Jun 09 12:00:25.763805 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763773 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 12:00:25.763870 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.763783 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 12:00:25.766365 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.766309 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 12:00:25.766365 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.766316 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:00:25.766527 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.766384 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:00:25.766527 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.766431 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 12:00:25.766612 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.766597 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-kube-api-access-st6lh" (OuterVolumeSpecName: "kube-api-access-st6lh") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "kube-api-access-st6lh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:00:25.772950 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.772929 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e8e742-ab29-499a-a61b-590221d1e221-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e8e8e742-ab29-499a-a61b-590221d1e221" (UID: "e8e8e742-ab29-499a-a61b-590221d1e221"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 12:00:25.864612 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864582 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8e8e742-ab29-499a-a61b-590221d1e221-ca-trust-extracted\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864612 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864605 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-image-registry-private-configuration\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864612 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864615 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-registry-certificates\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864818 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864624 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8e8e742-ab29-499a-a61b-590221d1e221-trusted-ca\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864818 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864632 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-registry-tls\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864818 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864640 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8e8e742-ab29-499a-a61b-590221d1e221-installation-pull-secrets\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864818 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864649 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st6lh\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-kube-api-access-st6lh\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.864818 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.864657 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8e8e742-ab29-499a-a61b-590221d1e221-bound-sa-token\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:00:25.956627 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:25.956590 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" podUID="b0696d5d-427c-4008-bd69-8bbd15a4eb25" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 09 12:00:26.458594 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.458561 2550 generic.go:356] "Generic (PLEG): container finished" podID="e8e8e742-ab29-499a-a61b-590221d1e221" containerID="09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72" exitCode=0 Jun 09 12:00:26.459067 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.458599 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" event={"ID":"e8e8e742-ab29-499a-a61b-590221d1e221","Type":"ContainerDied","Data":"09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72"} Jun 09 12:00:26.459067 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.458611 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" Jun 09 12:00:26.459067 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.458647 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcff6dcd4-4dqgw" event={"ID":"e8e8e742-ab29-499a-a61b-590221d1e221","Type":"ContainerDied","Data":"531d154b7cd49126a8322f2173e8711c82bef8e5ed679ead7dc659e4b74bf7c4"} Jun 09 12:00:26.459067 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.458674 2550 scope.go:117] "RemoveContainer" containerID="09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72" Jun 09 12:00:26.468562 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.468110 2550 scope.go:117] "RemoveContainer" containerID="09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72" Jun 09 12:00:26.469308 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:00:26.469269 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72\": container with ID starting with 09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72 not found: ID does not exist" containerID="09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72" Jun 09 12:00:26.469401 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.469313 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72"} err="failed to get container status \"09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72\": rpc error: code = NotFound desc = could not find container \"09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72\": container with ID starting with 09871f92f08caa2ebc43f277e8ae8f28ec36819db337a44d550cadd71b6eef72 not found: ID does not exist" Jun 09 12:00:26.496731 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.496708 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fcff6dcd4-4dqgw"] Jun 09 12:00:26.511894 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:26.511871 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fcff6dcd4-4dqgw"] Jun 09 12:00:27.200426 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:27.200393 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" path="/var/lib/kubelet/pods/e8e8e742-ab29-499a-a61b-590221d1e221/volumes" Jun 09 12:00:27.723166 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:27.723139 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:27.820568 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:27.820540 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:30.881636 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:30.881597 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:30.981603 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:30.981563 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:35.956329 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:35.956288 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" podUID="b0696d5d-427c-4008-bd69-8bbd15a4eb25" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 09 12:00:36.570115 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:36.570081 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:37.964013 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:37.963978 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:38.023081 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:38.023051 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:38.516916 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:38.516864 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:39.328898 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:39.328868 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:39.390129 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:39.390102 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:45.870912 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:45.870838 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:45.956280 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:45.956248 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" podUID="b0696d5d-427c-4008-bd69-8bbd15a4eb25" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 09 12:00:45.956418 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:45.956319 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" Jun 09 12:00:45.956774 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:45.956758 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"703ef050da75a42c05040c4c39b85f7737227b36b3bdd99fdb10675de4561983"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 09 12:00:45.956821 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:45.956792 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" podUID="b0696d5d-427c-4008-bd69-8bbd15a4eb25" containerName="service-proxy" containerID="cri-o://703ef050da75a42c05040c4c39b85f7737227b36b3bdd99fdb10675de4561983" gracePeriod=30 Jun 09 12:00:45.970728 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:45.970686 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:00:46.521756 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:46.521710 2550 generic.go:356] "Generic (PLEG): container finished" podID="b0696d5d-427c-4008-bd69-8bbd15a4eb25" containerID="703ef050da75a42c05040c4c39b85f7737227b36b3bdd99fdb10675de4561983" exitCode=2 Jun 09 12:00:46.521918 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:46.521786 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" event={"ID":"b0696d5d-427c-4008-bd69-8bbd15a4eb25","Type":"ContainerDied","Data":"703ef050da75a42c05040c4c39b85f7737227b36b3bdd99fdb10675de4561983"} Jun 09 12:00:46.521918 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:00:46.521830 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-645966b9cc-bgjdn" event={"ID":"b0696d5d-427c-4008-bd69-8bbd15a4eb25","Type":"ContainerStarted","Data":"e744cfdebb3b5c24a36dde1c761e061d193541b03fcc16245df7b6ee3a7bb59b"} Jun 09 12:01:04.145628 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.145592 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 12:01:04.147911 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.147885 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de637c06-0362-4c2d-b07f-af51cc52e6ce-metrics-certs\") pod \"network-metrics-daemon-j8vvj\" (UID: \"de637c06-0362-4c2d-b07f-af51cc52e6ce\") " pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 12:01:04.400204 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.400126 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wmll9\"" Jun 09 12:01:04.408056 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.408037 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j8vvj" Jun 09 12:01:04.531261 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.531229 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j8vvj"] Jun 09 12:01:04.534341 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:01:04.534312 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde637c06_0362_4c2d_b07f_af51cc52e6ce.slice/crio-c0cd84b11b2d414057f01954ecd416ed8a77fd07d2de875e50dc28ab9e5fd88a WatchSource:0}: Error finding container c0cd84b11b2d414057f01954ecd416ed8a77fd07d2de875e50dc28ab9e5fd88a: Status 404 returned error can't find the container with id c0cd84b11b2d414057f01954ecd416ed8a77fd07d2de875e50dc28ab9e5fd88a Jun 09 12:01:04.536181 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.536167 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 12:01:04.577831 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.577802 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j8vvj" event={"ID":"de637c06-0362-4c2d-b07f-af51cc52e6ce","Type":"ContainerStarted","Data":"c0cd84b11b2d414057f01954ecd416ed8a77fd07d2de875e50dc28ab9e5fd88a"} Jun 09 12:01:04.842448 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:04.842345 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:05.193975 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:05.193946 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:06.585564 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:06.585527 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j8vvj" event={"ID":"de637c06-0362-4c2d-b07f-af51cc52e6ce","Type":"ContainerStarted","Data":"8baf9efa7e3dc1ba8621469da9fa7d7ca1b3e4f176df1bb182b189383b838965"} Jun 09 12:01:06.585962 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:06.585591 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j8vvj" event={"ID":"de637c06-0362-4c2d-b07f-af51cc52e6ce","Type":"ContainerStarted","Data":"581d577551101da82a12a84305130b9c60d1df93904c21c1a0ea49ccf675fe48"} Jun 09 12:01:06.602438 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:06.602393 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j8vvj" podStartSLOduration=374.632014189 podStartE2EDuration="6m15.602378996s" totalImagesPullingTime="970.364807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:01:06.602252445 +0000 UTC m=+375.884069669" watchObservedRunningTime="2026-06-09 12:01:06.602378996 +0000 UTC m=+375.884196219" Jun 09 12:01:06.690996 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:06.690965 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:06.795336 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:06.795310 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:14.427303 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:14.427270 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:14.493010 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:14.492981 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:17.754070 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:17.754031 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:20.403132 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:20.403094 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:20.470412 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:20.470373 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:24.953136 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:24.953092 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:25.005942 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:25.005894 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:01:25.065299 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:01:25.065259 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:20.140977 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:20.140893 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:20.216089 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:20.216060 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:28.001930 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:28.001892 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:32.009942 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:32.009896 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:32.105815 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:32.105786 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:35.585680 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:35.585636 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:35.652003 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:35.651975 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:45.902758 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:45.902704 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:46.002331 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:46.002287 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:46.077176 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:46.077151 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:46.315774 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:46.315662 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:46.415717 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:46.415685 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:02:46.480285 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:02:46.480252 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:03:23.158239 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:03:23.158204 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:03:31.042050 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:03:31.042011 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:03:31.140823 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:03:31.140787 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:03:57.815006 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:03:57.814972 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:03:58.765063 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:03:58.765020 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:01.558964 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:01.558929 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:01.658316 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:01.658272 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:03.152484 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:03.152453 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:03.466953 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:03.466877 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:04.579944 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.579912 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j"] Jun 09 12:04:04.580311 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.580124 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" containerName="registry" Jun 09 12:04:04.580311 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.580134 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" containerName="registry" Jun 09 12:04:04.580311 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.580183 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8e8e742-ab29-499a-a61b-590221d1e221" containerName="registry" Jun 09 12:04:04.581670 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.581654 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.584124 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.584103 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 09 12:04:04.584223 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.584138 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 09 12:04:04.584927 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.584910 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-n246d\"" Jun 09 12:04:04.597291 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.597269 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j"] Jun 09 12:04:04.635234 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.635205 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31e156a2-5c1a-44aa-b058-b2014d0a56b5-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xwq6j\" (UID: \"31e156a2-5c1a-44aa-b058-b2014d0a56b5\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.635340 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.635252 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdt2k\" (UniqueName: \"kubernetes.io/projected/31e156a2-5c1a-44aa-b058-b2014d0a56b5-kube-api-access-jdt2k\") pod \"cert-manager-webhook-7dc76cbb94-xwq6j\" (UID: \"31e156a2-5c1a-44aa-b058-b2014d0a56b5\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.736573 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.736529 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31e156a2-5c1a-44aa-b058-b2014d0a56b5-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xwq6j\" (UID: \"31e156a2-5c1a-44aa-b058-b2014d0a56b5\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.736722 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.736594 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdt2k\" (UniqueName: \"kubernetes.io/projected/31e156a2-5c1a-44aa-b058-b2014d0a56b5-kube-api-access-jdt2k\") pod \"cert-manager-webhook-7dc76cbb94-xwq6j\" (UID: \"31e156a2-5c1a-44aa-b058-b2014d0a56b5\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.756052 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.756025 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdt2k\" (UniqueName: \"kubernetes.io/projected/31e156a2-5c1a-44aa-b058-b2014d0a56b5-kube-api-access-jdt2k\") pod \"cert-manager-webhook-7dc76cbb94-xwq6j\" (UID: \"31e156a2-5c1a-44aa-b058-b2014d0a56b5\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.758434 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.758408 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31e156a2-5c1a-44aa-b058-b2014d0a56b5-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xwq6j\" (UID: \"31e156a2-5c1a-44aa-b058-b2014d0a56b5\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:04.890683 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:04.890654 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:05.014625 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:05.014585 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j"] Jun 09 12:04:05.017642 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:04:05.017613 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31e156a2_5c1a_44aa_b058_b2014d0a56b5.slice/crio-2e1e4333d2011430919fa05933194d0652ef5f412404f465c8eb9366736d422b WatchSource:0}: Error finding container 2e1e4333d2011430919fa05933194d0652ef5f412404f465c8eb9366736d422b: Status 404 returned error can't find the container with id 2e1e4333d2011430919fa05933194d0652ef5f412404f465c8eb9366736d422b Jun 09 12:04:05.139748 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:05.139712 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" event={"ID":"31e156a2-5c1a-44aa-b058-b2014d0a56b5","Type":"ContainerStarted","Data":"2e1e4333d2011430919fa05933194d0652ef5f412404f465c8eb9366736d422b"} Jun 09 12:04:06.482523 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:06.482486 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:06.719125 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:06.719086 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:08.151861 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:08.151821 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" event={"ID":"31e156a2-5c1a-44aa-b058-b2014d0a56b5","Type":"ContainerStarted","Data":"4358595e22e71bf6d8db431d15d89957a52f8093e164345f8588c4ad8751967a"} Jun 09 12:04:08.152301 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:08.152044 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:08.174407 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:08.174363 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" podStartSLOduration=1.8715567979999999 podStartE2EDuration="4.174350511s" totalImagesPullingTime="2.302793713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:04:08.173430436 +0000 UTC m=+557.455247659" watchObservedRunningTime="2026-06-09 12:04:08.174350511 +0000 UTC m=+557.456167734" Jun 09 12:04:08.363749 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:08.363715 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:08.442488 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:08.442418 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:12.919427 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:12.919388 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:13.020805 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:13.020777 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:14.157496 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:14.157470 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xwq6j" Jun 09 12:04:14.183313 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:14.183282 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:14.295814 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:14.295773 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:14.919556 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:14.919524 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:20.749298 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:20.749264 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:20.859901 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:20.859869 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:22.878506 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:22.878472 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:24.107960 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.107922 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-wmj7c"] Jun 09 12:04:24.109910 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.109889 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.112227 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.112209 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g4646\"" Jun 09 12:04:24.119463 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.119443 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-wmj7c"] Jun 09 12:04:24.159980 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.159954 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd7nk\" (UniqueName: \"kubernetes.io/projected/ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601-kube-api-access-nd7nk\") pod \"cert-manager-59c77684cc-wmj7c\" (UID: \"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601\") " pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.159980 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.159981 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601-bound-sa-token\") pod \"cert-manager-59c77684cc-wmj7c\" (UID: \"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601\") " pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.260383 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.260362 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd7nk\" (UniqueName: \"kubernetes.io/projected/ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601-kube-api-access-nd7nk\") pod \"cert-manager-59c77684cc-wmj7c\" (UID: \"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601\") " pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.260495 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.260390 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601-bound-sa-token\") pod \"cert-manager-59c77684cc-wmj7c\" (UID: \"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601\") " pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.268212 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.268180 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601-bound-sa-token\") pod \"cert-manager-59c77684cc-wmj7c\" (UID: \"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601\") " pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.268420 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.268399 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd7nk\" (UniqueName: \"kubernetes.io/projected/ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601-kube-api-access-nd7nk\") pod \"cert-manager-59c77684cc-wmj7c\" (UID: \"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601\") " pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.418248 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.418220 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-wmj7c" Jun 09 12:04:24.537280 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:24.537250 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-wmj7c"] Jun 09 12:04:24.540955 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:04:24.540925 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae28e02b_5fb2_4b3e_a1f1_1c9d1df62601.slice/crio-37621a1d38329d15abe51a4ebcd6add3b6b0cba7690d8f1cab9f38b666dfcdcd WatchSource:0}: Error finding container 37621a1d38329d15abe51a4ebcd6add3b6b0cba7690d8f1cab9f38b666dfcdcd: Status 404 returned error can't find the container with id 37621a1d38329d15abe51a4ebcd6add3b6b0cba7690d8f1cab9f38b666dfcdcd Jun 09 12:04:25.209278 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:25.209244 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-wmj7c" event={"ID":"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601","Type":"ContainerStarted","Data":"20b3371170bd9941648444b81c2df3463a998fa111af1ac8c801fc83defc94f4"} Jun 09 12:04:25.209657 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:25.209284 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-wmj7c" event={"ID":"ae28e02b-5fb2-4b3e-a1f1-1c9d1df62601","Type":"ContainerStarted","Data":"37621a1d38329d15abe51a4ebcd6add3b6b0cba7690d8f1cab9f38b666dfcdcd"} Jun 09 12:04:25.226692 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:25.226626 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-wmj7c" podStartSLOduration=1.226611846 podStartE2EDuration="1.226611846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:04:25.225058597 +0000 UTC m=+574.506875820" watchObservedRunningTime="2026-06-09 12:04:25.226611846 +0000 UTC m=+574.508429068" Jun 09 12:04:39.488832 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:39.488783 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:39.596886 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:39.596855 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:39.998337 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:39.998307 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:40.236193 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.236159 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:40.418774 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.418726 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:40.540896 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.540869 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk"] Jun 09 12:04:40.542748 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.542718 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.545942 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.545919 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 12:04:40.546215 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.546198 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nf72c\"" Jun 09 12:04:40.546401 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.546385 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 12:04:40.546477 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.546425 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 12:04:40.546477 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.546444 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 12:04:40.560639 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.560615 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk"] Jun 09 12:04:40.674374 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.674288 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6k9\" (UniqueName: \"kubernetes.io/projected/b0df865d-9ca9-489b-9a45-d7718ebb26fe-kube-api-access-ql6k9\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.674374 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.674335 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0df865d-9ca9-489b-9a45-d7718ebb26fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.674374 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.674368 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0df865d-9ca9-489b-9a45-d7718ebb26fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.674578 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.674395 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0df865d-9ca9-489b-9a45-d7718ebb26fe-manifests\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.674578 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.674411 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0df865d-9ca9-489b-9a45-d7718ebb26fe-tmp\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.724544 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.724509 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:40.775339 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.775305 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6k9\" (UniqueName: \"kubernetes.io/projected/b0df865d-9ca9-489b-9a45-d7718ebb26fe-kube-api-access-ql6k9\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.775494 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.775354 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0df865d-9ca9-489b-9a45-d7718ebb26fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.775494 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.775382 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0df865d-9ca9-489b-9a45-d7718ebb26fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.775494 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.775408 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0df865d-9ca9-489b-9a45-d7718ebb26fe-manifests\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.775494 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.775423 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0df865d-9ca9-489b-9a45-d7718ebb26fe-tmp\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.775926 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.775905 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0df865d-9ca9-489b-9a45-d7718ebb26fe-manifests\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.777659 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.777635 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0df865d-9ca9-489b-9a45-d7718ebb26fe-tmp\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.777813 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.777793 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0df865d-9ca9-489b-9a45-d7718ebb26fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.777863 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.777850 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0df865d-9ca9-489b-9a45-d7718ebb26fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.789635 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.789615 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6k9\" (UniqueName: \"kubernetes.io/projected/b0df865d-9ca9-489b-9a45-d7718ebb26fe-kube-api-access-ql6k9\") pod \"opendatahub-operator-controller-manager-86976b546f-ljjtk\" (UID: \"b0df865d-9ca9-489b-9a45-d7718ebb26fe\") " pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.821608 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.821583 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:40.853065 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.853044 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:40.923652 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.923537 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:40.995249 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:40.995219 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk"] Jun 09 12:04:40.998399 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:04:40.998375 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0df865d_9ca9_489b_9a45_d7718ebb26fe.slice/crio-14f221907c1309c56e1788238239cb648408513cc439ebec634bdd7d5356261d WatchSource:0}: Error finding container 14f221907c1309c56e1788238239cb648408513cc439ebec634bdd7d5356261d: Status 404 returned error can't find the container with id 14f221907c1309c56e1788238239cb648408513cc439ebec634bdd7d5356261d Jun 09 12:04:41.003792 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:41.003766 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:41.076896 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:41.076869 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:41.177508 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:41.177481 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:41.260450 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:41.260382 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" event={"ID":"b0df865d-9ca9-489b-9a45-d7718ebb26fe","Type":"ContainerStarted","Data":"14f221907c1309c56e1788238239cb648408513cc439ebec634bdd7d5356261d"} Jun 09 12:04:41.278166 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:41.278142 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:44.276085 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:44.276048 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" event={"ID":"b0df865d-9ca9-489b-9a45-d7718ebb26fe","Type":"ContainerStarted","Data":"bae18e1269fc47b0a49769562e5ee373b6520924066a8e6683170e06827de7f5"} Jun 09 12:04:45.285286 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:45.285194 2550 generic.go:356] "Generic (PLEG): container finished" podID="b0df865d-9ca9-489b-9a45-d7718ebb26fe" containerID="bae18e1269fc47b0a49769562e5ee373b6520924066a8e6683170e06827de7f5" exitCode=0 Jun 09 12:04:45.285286 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:45.285265 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" event={"ID":"b0df865d-9ca9-489b-9a45-d7718ebb26fe","Type":"ContainerDied","Data":"bae18e1269fc47b0a49769562e5ee373b6520924066a8e6683170e06827de7f5"} Jun 09 12:04:46.291624 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:46.291587 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" event={"ID":"b0df865d-9ca9-489b-9a45-d7718ebb26fe","Type":"ContainerStarted","Data":"9f250ae706a775f58faf294d12835771b95abc2e7cbd7e3b1415010eb444bf50"} Jun 09 12:04:46.292041 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:46.291723 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:46.335689 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:46.335641 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" podStartSLOduration=2.2297369160000002 podStartE2EDuration="6.335626409s" totalImagesPullingTime="3.132787293s" totalInitContainerRuntime="973.1022ms" isStatefulPod=false podCreationTimestamp="2026-06-09 12:04:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:04:46.333642816 +0000 UTC m=+595.615460279" watchObservedRunningTime="2026-06-09 12:04:46.335626409 +0000 UTC m=+595.617443632" Jun 09 12:04:51.560834 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:51.560794 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:51.689678 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:51.689649 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:53.265626 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:53.265596 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:53.442865 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:53.442831 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:55.214145 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:55.214114 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:55.313188 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:55.313156 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:55.685880 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:55.685852 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:55.784377 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:55.784345 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:04:57.296971 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:57.296938 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86976b546f-ljjtk" Jun 09 12:04:59.610038 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:04:59.610004 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:00.293186 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.293150 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fd697648d-bvngf"] Jun 09 12:05:00.296273 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.296258 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.302488 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.302317 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 09 12:05:00.302488 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.302337 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 12:05:00.302617 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.302608 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lnh2f\"" Jun 09 12:05:00.302801 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.302780 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 12:05:00.303054 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.303039 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 09 12:05:00.327076 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.327052 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fd697648d-bvngf"] Jun 09 12:05:00.420004 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.419976 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2dn9\" (UniqueName: \"kubernetes.io/projected/06fe0e99-8910-4c42-a666-55e73b2b6347-kube-api-access-m2dn9\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.420135 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.420012 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06fe0e99-8910-4c42-a666-55e73b2b6347-tls-certs\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.420135 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.420044 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06fe0e99-8910-4c42-a666-55e73b2b6347-tmp\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.520877 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.520853 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06fe0e99-8910-4c42-a666-55e73b2b6347-tmp\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.520976 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.520894 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2dn9\" (UniqueName: \"kubernetes.io/projected/06fe0e99-8910-4c42-a666-55e73b2b6347-kube-api-access-m2dn9\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.520976 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.520918 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06fe0e99-8910-4c42-a666-55e73b2b6347-tls-certs\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.521059 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:05:00.520998 2550 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 09 12:05:00.521059 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:05:00.521054 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06fe0e99-8910-4c42-a666-55e73b2b6347-tls-certs podName:06fe0e99-8910-4c42-a666-55e73b2b6347 nodeName:}" failed. No retries permitted until 2026-06-09 12:05:01.021038519 +0000 UTC m=+610.302855719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/06fe0e99-8910-4c42-a666-55e73b2b6347-tls-certs") pod "kube-auth-proxy-6fd697648d-bvngf" (UID: "06fe0e99-8910-4c42-a666-55e73b2b6347") : secret "kube-auth-proxy-tls" not found Jun 09 12:05:00.523070 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.523052 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06fe0e99-8910-4c42-a666-55e73b2b6347-tmp\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:00.537657 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:00.537635 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2dn9\" (UniqueName: \"kubernetes.io/projected/06fe0e99-8910-4c42-a666-55e73b2b6347-kube-api-access-m2dn9\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:01.023989 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.023956 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06fe0e99-8910-4c42-a666-55e73b2b6347-tls-certs\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:01.026382 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.026363 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06fe0e99-8910-4c42-a666-55e73b2b6347-tls-certs\") pod \"kube-auth-proxy-6fd697648d-bvngf\" (UID: \"06fe0e99-8910-4c42-a666-55e73b2b6347\") " pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:01.077554 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.077517 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:01.205328 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.205292 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" Jun 09 12:05:01.348041 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.348011 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fd697648d-bvngf"] Jun 09 12:05:01.350570 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:05:01.350540 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fe0e99_8910_4c42_a666_55e73b2b6347.slice/crio-77590d2533ea7df861b423c27a3ee35448f4caa09ffcfc8aa40c1abf321b2b6f WatchSource:0}: Error finding container 77590d2533ea7df861b423c27a3ee35448f4caa09ffcfc8aa40c1abf321b2b6f: Status 404 returned error can't find the container with id 77590d2533ea7df861b423c27a3ee35448f4caa09ffcfc8aa40c1abf321b2b6f Jun 09 12:05:01.374944 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.374920 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:01.974976 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:01.974936 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:02.344944 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:02.344860 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" event={"ID":"06fe0e99-8910-4c42-a666-55e73b2b6347","Type":"ContainerStarted","Data":"77590d2533ea7df861b423c27a3ee35448f4caa09ffcfc8aa40c1abf321b2b6f"} Jun 09 12:05:05.357258 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:05.357219 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" event={"ID":"06fe0e99-8910-4c42-a666-55e73b2b6347","Type":"ContainerStarted","Data":"38d41bcc19923dff6a28c4010fb11222e1d8df5ed87c406b767b21b97c5b87cc"} Jun 09 12:05:05.374786 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:05.374722 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fd697648d-bvngf" podStartSLOduration=2.258764375 podStartE2EDuration="5.374709719s" totalImagesPullingTime="3.115945344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:05:05.373879191 +0000 UTC m=+614.655696415" watchObservedRunningTime="2026-06-09 12:05:05.374709719 +0000 UTC m=+614.656526942" Jun 09 12:05:08.111993 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.111958 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv"] Jun 09 12:05:08.115148 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.115132 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.118049 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.118027 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 09 12:05:08.118678 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.118662 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 09 12:05:08.119027 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.119013 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 09 12:05:08.119103 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.119065 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 09 12:05:08.119167 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.119067 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 09 12:05:08.119784 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.119769 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pr6dx\"" Jun 09 12:05:08.128294 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.128271 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv"] Jun 09 12:05:08.278328 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.278290 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ae71faa1-1a03-4d63-95f6-f047e24b79b1-metrics-cert\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.278328 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.278340 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ae71faa1-1a03-4d63-95f6-f047e24b79b1-manager-config\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.278525 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.278360 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kjmz\" (UniqueName: \"kubernetes.io/projected/ae71faa1-1a03-4d63-95f6-f047e24b79b1-kube-api-access-7kjmz\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.278525 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.278381 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae71faa1-1a03-4d63-95f6-f047e24b79b1-cert\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.379450 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.379418 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae71faa1-1a03-4d63-95f6-f047e24b79b1-cert\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.379605 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.379486 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ae71faa1-1a03-4d63-95f6-f047e24b79b1-metrics-cert\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.379605 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.379505 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ae71faa1-1a03-4d63-95f6-f047e24b79b1-manager-config\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.379605 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.379527 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kjmz\" (UniqueName: \"kubernetes.io/projected/ae71faa1-1a03-4d63-95f6-f047e24b79b1-kube-api-access-7kjmz\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.380209 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.380188 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ae71faa1-1a03-4d63-95f6-f047e24b79b1-manager-config\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.381978 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.381958 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ae71faa1-1a03-4d63-95f6-f047e24b79b1-metrics-cert\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.382064 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.382021 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae71faa1-1a03-4d63-95f6-f047e24b79b1-cert\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.388184 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.388164 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kjmz\" (UniqueName: \"kubernetes.io/projected/ae71faa1-1a03-4d63-95f6-f047e24b79b1-kube-api-access-7kjmz\") pod \"lws-controller-manager-57f7d46b46-rn8gv\" (UID: \"ae71faa1-1a03-4d63-95f6-f047e24b79b1\") " pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.424195 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.424173 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:08.562389 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:08.562362 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv"] Jun 09 12:05:08.565753 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:05:08.565711 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae71faa1_1a03_4d63_95f6_f047e24b79b1.slice/crio-105457d18d059ca95890f1e026ed49fddb68591aee35b6e1200085e455c78878 WatchSource:0}: Error finding container 105457d18d059ca95890f1e026ed49fddb68591aee35b6e1200085e455c78878: Status 404 returned error can't find the container with id 105457d18d059ca95890f1e026ed49fddb68591aee35b6e1200085e455c78878 Jun 09 12:05:09.374318 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:09.374278 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" event={"ID":"ae71faa1-1a03-4d63-95f6-f047e24b79b1","Type":"ContainerStarted","Data":"105457d18d059ca95890f1e026ed49fddb68591aee35b6e1200085e455c78878"} Jun 09 12:05:11.203788 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:11.203645 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:11.324800 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:11.324769 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:11.604978 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:11.604954 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:11.725029 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:11.724996 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:11.844357 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:11.844287 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:12.391639 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:12.391605 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" event={"ID":"ae71faa1-1a03-4d63-95f6-f047e24b79b1","Type":"ContainerStarted","Data":"a2a4dda8016ea076227c588126fd18b50f5aca585482ec5d2d055a96780e09b9"} Jun 09 12:05:12.392020 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:12.391758 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:12.411005 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:12.410960 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" podStartSLOduration=1.426473556 podStartE2EDuration="4.410947462s" totalImagesPullingTime="2.984473906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:05:12.410190354 +0000 UTC m=+621.692007581" watchObservedRunningTime="2026-06-09 12:05:12.410947462 +0000 UTC m=+621.692764685" Jun 09 12:05:14.496419 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:14.496389 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:14.590238 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:14.590212 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:16.658505 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:16.658463 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:16.863358 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:16.863324 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:19.953465 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:19.953431 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:20.107923 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:20.107886 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:20.265560 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:20.265475 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:20.511324 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:20.511290 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:20.884333 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:20.884304 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:23.397421 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:23.397396 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57f7d46b46-rn8gv" Jun 09 12:05:31.008146 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:31.008113 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:31.122308 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:31.122279 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:31.395588 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:31.395563 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:36.229955 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:36.229923 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:47.964074 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:47.964040 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:48.065242 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:48.065207 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:50.120050 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:50.120012 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:50.181635 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:50.181607 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:50.795804 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:50.795767 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:50.912439 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:50.912403 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:51.342786 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:51.342750 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:54.664557 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:54.664524 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:54.765361 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:54.765330 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:54.978486 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:54.978416 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:55.265372 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:55.265290 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:55.412439 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:55.412402 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:05:55.472208 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:05:55.472182 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:01.120727 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:01.120689 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:01.222855 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:01.222826 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:01.412170 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:01.412136 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:21.201694 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:21.201657 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:21.270243 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:21.270212 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:28.901203 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:28.901169 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:29.551886 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:29.551852 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:30.567543 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:30.567498 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:30.679429 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:30.679398 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:32.655904 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:32.655872 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:32.759702 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:32.759671 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:32.855628 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:32.855598 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:34.155921 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:34.155891 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:36.331951 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:36.331920 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:36.656836 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:36.656802 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:37.766285 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:37.766255 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:40.716286 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:40.716251 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:41.394284 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:41.394254 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:43.630781 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:43.630747 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:43.736917 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:43.736882 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:43.855838 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:43.855806 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:47.343992 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:47.343956 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:47.464806 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:47.464773 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:50.807622 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:50.807590 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:51.022417 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:51.022385 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:52.016027 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:52.015994 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:52.112798 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:52.112765 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:59.052558 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:59.052522 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:06:59.156428 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:06:59.156394 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:01.317914 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:01.317878 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:03.252784 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:03.252752 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:03.384527 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:03.384501 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:03.748693 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:03.748661 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:03.885960 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:03.885928 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:07.422685 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:07.422652 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:07.524413 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:07.524384 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:07.735313 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:07.735234 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:11.147817 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:11.147787 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:11.261998 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:11.261970 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:18.106241 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:18.106200 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:18.212803 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:18.212772 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:44.307207 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:44.307171 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:44.407122 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:44.407085 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:47.004271 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:47.004237 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:47.078153 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:47.078128 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:48.011577 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:48.011550 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:48.420022 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:48.419991 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:48.602811 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:48.602783 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:48.698860 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:48.698783 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:49.345457 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.345425 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-pbrsb"] Jun 09 12:07:49.348380 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.348361 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:49.354583 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.354453 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 12:07:49.354749 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.354714 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pj99j\"" Jun 09 12:07:49.355264 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.355241 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 12:07:49.361574 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.361542 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbrsb"] Jun 09 12:07:49.389418 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.389396 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:49.457327 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.457302 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69d9q\" (UniqueName: \"kubernetes.io/projected/ad317055-4fa1-42a9-9592-bd1749b11d65-kube-api-access-69d9q\") pod \"authorino-58cd955675-pbrsb\" (UID: \"ad317055-4fa1-42a9-9592-bd1749b11d65\") " pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:49.491621 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.491590 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:07:49.558247 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.558211 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69d9q\" (UniqueName: \"kubernetes.io/projected/ad317055-4fa1-42a9-9592-bd1749b11d65-kube-api-access-69d9q\") pod \"authorino-58cd955675-pbrsb\" (UID: \"ad317055-4fa1-42a9-9592-bd1749b11d65\") " pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:49.569769 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.569722 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69d9q\" (UniqueName: \"kubernetes.io/projected/ad317055-4fa1-42a9-9592-bd1749b11d65-kube-api-access-69d9q\") pod \"authorino-58cd955675-pbrsb\" (UID: \"ad317055-4fa1-42a9-9592-bd1749b11d65\") " pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:49.657401 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.657374 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:49.661565 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.661547 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kltn8"] Jun 09 12:07:49.664638 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.664624 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:07:49.674049 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.674021 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kltn8"] Jun 09 12:07:49.759369 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.759335 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krhcg\" (UniqueName: \"kubernetes.io/projected/5cd95749-5690-49a9-8d2f-1ec1a536bee8-kube-api-access-krhcg\") pod \"authorino-75d556bd77-kltn8\" (UID: \"5cd95749-5690-49a9-8d2f-1ec1a536bee8\") " pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:07:49.780209 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.780182 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbrsb"] Jun 09 12:07:49.782688 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:07:49.782663 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad317055_4fa1_42a9_9592_bd1749b11d65.slice/crio-55c87affbab016a78218783a379916f574c463e6dfb20e5625ba04a0649f1ecd WatchSource:0}: Error finding container 55c87affbab016a78218783a379916f574c463e6dfb20e5625ba04a0649f1ecd: Status 404 returned error can't find the container with id 55c87affbab016a78218783a379916f574c463e6dfb20e5625ba04a0649f1ecd Jun 09 12:07:49.784139 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.784118 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 12:07:49.860369 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.860347 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krhcg\" (UniqueName: \"kubernetes.io/projected/5cd95749-5690-49a9-8d2f-1ec1a536bee8-kube-api-access-krhcg\") pod \"authorino-75d556bd77-kltn8\" (UID: \"5cd95749-5690-49a9-8d2f-1ec1a536bee8\") " pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:07:49.869116 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.869097 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krhcg\" (UniqueName: \"kubernetes.io/projected/5cd95749-5690-49a9-8d2f-1ec1a536bee8-kube-api-access-krhcg\") pod \"authorino-75d556bd77-kltn8\" (UID: \"5cd95749-5690-49a9-8d2f-1ec1a536bee8\") " pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:07:49.935325 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.935266 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbrsb" event={"ID":"ad317055-4fa1-42a9-9592-bd1749b11d65","Type":"ContainerStarted","Data":"55c87affbab016a78218783a379916f574c463e6dfb20e5625ba04a0649f1ecd"} Jun 09 12:07:49.985801 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:49.985777 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:07:50.108628 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:50.108597 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kltn8"] Jun 09 12:07:50.111698 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:07:50.111676 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cd95749_5690_49a9_8d2f_1ec1a536bee8.slice/crio-e3b92d27055df549f5ab24a3284ab9ba48c53052715c1c91a7d0707628759cd8 WatchSource:0}: Error finding container e3b92d27055df549f5ab24a3284ab9ba48c53052715c1c91a7d0707628759cd8: Status 404 returned error can't find the container with id e3b92d27055df549f5ab24a3284ab9ba48c53052715c1c91a7d0707628759cd8 Jun 09 12:07:50.943310 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:50.943268 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kltn8" event={"ID":"5cd95749-5690-49a9-8d2f-1ec1a536bee8","Type":"ContainerStarted","Data":"e3b92d27055df549f5ab24a3284ab9ba48c53052715c1c91a7d0707628759cd8"} Jun 09 12:07:53.956768 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:53.956719 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbrsb" event={"ID":"ad317055-4fa1-42a9-9592-bd1749b11d65","Type":"ContainerStarted","Data":"f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc"} Jun 09 12:07:53.958042 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:53.958013 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kltn8" event={"ID":"5cd95749-5690-49a9-8d2f-1ec1a536bee8","Type":"ContainerStarted","Data":"7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2"} Jun 09 12:07:53.992496 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:53.992444 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-pbrsb" podStartSLOduration=1.283048715 podStartE2EDuration="4.992424607s" totalImagesPullingTime="3.709375892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:07:53.973382064 +0000 UTC m=+783.255199288" watchObservedRunningTime="2026-06-09 12:07:53.992424607 +0000 UTC m=+783.274241835" Jun 09 12:07:53.993529 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:53.993490 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kltn8" podStartSLOduration=1.7958094089999999 podStartE2EDuration="4.993478732s" totalImagesPullingTime="3.197669323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:07:53.991696575 +0000 UTC m=+783.273513799" watchObservedRunningTime="2026-06-09 12:07:53.993478732 +0000 UTC m=+783.275295957" Jun 09 12:07:54.030138 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:54.030108 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbrsb"] Jun 09 12:07:55.967259 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:55.967220 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-pbrsb" podUID="ad317055-4fa1-42a9-9592-bd1749b11d65" containerName="authorino" containerID="cri-o://f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc" gracePeriod=30 Jun 09 12:07:56.204814 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.204792 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:56.315442 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.315378 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69d9q\" (UniqueName: \"kubernetes.io/projected/ad317055-4fa1-42a9-9592-bd1749b11d65-kube-api-access-69d9q\") pod \"ad317055-4fa1-42a9-9592-bd1749b11d65\" (UID: \"ad317055-4fa1-42a9-9592-bd1749b11d65\") " Jun 09 12:07:56.317462 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.317437 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad317055-4fa1-42a9-9592-bd1749b11d65-kube-api-access-69d9q" (OuterVolumeSpecName: "kube-api-access-69d9q") pod "ad317055-4fa1-42a9-9592-bd1749b11d65" (UID: "ad317055-4fa1-42a9-9592-bd1749b11d65"). InnerVolumeSpecName "kube-api-access-69d9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:07:56.416650 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.416621 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69d9q\" (UniqueName: \"kubernetes.io/projected/ad317055-4fa1-42a9-9592-bd1749b11d65-kube-api-access-69d9q\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:07:56.971274 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.971239 2550 generic.go:356] "Generic (PLEG): container finished" podID="ad317055-4fa1-42a9-9592-bd1749b11d65" containerID="f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc" exitCode=0 Jun 09 12:07:56.971718 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.971283 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbrsb" Jun 09 12:07:56.971718 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.971321 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbrsb" event={"ID":"ad317055-4fa1-42a9-9592-bd1749b11d65","Type":"ContainerDied","Data":"f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc"} Jun 09 12:07:56.971718 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.971364 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbrsb" event={"ID":"ad317055-4fa1-42a9-9592-bd1749b11d65","Type":"ContainerDied","Data":"55c87affbab016a78218783a379916f574c463e6dfb20e5625ba04a0649f1ecd"} Jun 09 12:07:56.971718 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.971381 2550 scope.go:117] "RemoveContainer" containerID="f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc" Jun 09 12:07:56.979399 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.979386 2550 scope.go:117] "RemoveContainer" containerID="f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc" Jun 09 12:07:56.979646 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:07:56.979623 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc\": container with ID starting with f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc not found: ID does not exist" containerID="f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc" Jun 09 12:07:56.979696 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.979655 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc"} err="failed to get container status \"f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc\": rpc error: code = NotFound desc = could not find container \"f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc\": container with ID starting with f61e62b6b98889ccd82fd9026c6ee3fecc266d7f5dc0f07bd2d245f6cbf9f6fc not found: ID does not exist" Jun 09 12:07:56.991701 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.991681 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbrsb"] Jun 09 12:07:56.995580 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:56.995561 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbrsb"] Jun 09 12:07:57.200490 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:07:57.200460 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad317055-4fa1-42a9-9592-bd1749b11d65" path="/var/lib/kubelet/pods/ad317055-4fa1-42a9-9592-bd1749b11d65/volumes" Jun 09 12:08:04.506028 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:04.505996 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:14.315512 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:14.315431 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:14.411447 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:14.411417 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:14.574380 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:14.574302 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:20.360152 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:20.360119 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:39.772768 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:39.772717 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:39.872912 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:39.872882 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:08:40.172338 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.172308 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:08:40.172560 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.172548 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad317055-4fa1-42a9-9592-bd1749b11d65" containerName="authorino" Jun 09 12:08:40.172605 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.172561 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad317055-4fa1-42a9-9592-bd1749b11d65" containerName="authorino" Jun 09 12:08:40.172638 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.172613 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad317055-4fa1-42a9-9592-bd1749b11d65" containerName="authorino" Jun 09 12:08:40.178052 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.178032 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:40.180426 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.180404 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 12:08:40.180655 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.180635 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wq76x\"" Jun 09 12:08:40.181288 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.181269 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 12:08:40.181385 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.181274 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 12:08:40.187584 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.187564 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:08:40.207151 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.207131 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxg6q\" (UniqueName: \"kubernetes.io/projected/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16-kube-api-access-pxg6q\") pod \"maas-keycloak-0\" (UID: \"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:40.308102 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.308077 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxg6q\" (UniqueName: \"kubernetes.io/projected/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16-kube-api-access-pxg6q\") pod \"maas-keycloak-0\" (UID: \"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:40.316490 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.316465 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxg6q\" (UniqueName: \"kubernetes.io/projected/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16-kube-api-access-pxg6q\") pod \"maas-keycloak-0\" (UID: \"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:40.488286 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.488213 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:40.627410 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:40.627373 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:08:40.630193 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:08:40.630167 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab25a28a_aae4_4f7f_9ed3_df1faa6afb16.slice/crio-3456fd79daca692b461a717121106328e99de9c0414eb8274b3881bbbf7d77da WatchSource:0}: Error finding container 3456fd79daca692b461a717121106328e99de9c0414eb8274b3881bbbf7d77da: Status 404 returned error can't find the container with id 3456fd79daca692b461a717121106328e99de9c0414eb8274b3881bbbf7d77da Jun 09 12:08:41.129821 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:41.129781 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16","Type":"ContainerStarted","Data":"3456fd79daca692b461a717121106328e99de9c0414eb8274b3881bbbf7d77da"} Jun 09 12:08:47.159863 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:47.159819 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16","Type":"ContainerStarted","Data":"17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459"} Jun 09 12:08:47.185525 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:47.185460 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.6435864119999999 podStartE2EDuration="7.185439036s" totalImagesPullingTime="5.541852624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:08:47.184172716 +0000 UTC m=+836.465989942" watchObservedRunningTime="2026-06-09 12:08:47.185439036 +0000 UTC m=+836.467256260" Jun 09 12:08:47.488604 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:47.488501 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:47.490604 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:47.490564 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:48.489149 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:48.489110 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:49.489484 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:49.489430 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:50.489368 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:50.488623 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:50.489368 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:50.488970 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:51.489781 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:51.489229 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:52.488722 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:52.488676 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:53.489275 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:53.489216 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:54.489144 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:54.489094 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:55.488819 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:55.488765 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:56.489341 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:56.489279 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:57.489072 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:57.489000 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:58.488800 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:58.488758 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.20:9000/health/started\": dial tcp 10.132.0.20:9000: connect: connection refused" Jun 09 12:08:59.617385 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:59.617353 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 12:08:59.634791 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:08:59.634749 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 12:09:09.623224 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:09.623193 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 12:09:10.279234 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.279198 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:10.484238 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.484208 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:10.506381 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.506351 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdnfm"] Jun 09 12:09:10.517307 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.517285 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdnfm"] Jun 09 12:09:10.517424 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.517365 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:10.664772 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.664721 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfd9c\" (UniqueName: \"kubernetes.io/projected/69986027-fdcf-4e90-bca7-7ecbe1c407c1-kube-api-access-lfd9c\") pod \"authorino-5d4b67c446-sdnfm\" (UID: \"69986027-fdcf-4e90-bca7-7ecbe1c407c1\") " pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:10.724794 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.724760 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdnfm"] Jun 09 12:09:10.724984 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:10.724966 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lfd9c], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-sdnfm" podUID="69986027-fdcf-4e90-bca7-7ecbe1c407c1" Jun 09 12:09:10.747429 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.747396 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58668d47d9-clmd2"] Jun 09 12:09:10.750586 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.750566 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:10.757698 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.757672 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58668d47d9-clmd2"] Jun 09 12:09:10.765242 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.765216 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfd9c\" (UniqueName: \"kubernetes.io/projected/69986027-fdcf-4e90-bca7-7ecbe1c407c1-kube-api-access-lfd9c\") pod \"authorino-5d4b67c446-sdnfm\" (UID: \"69986027-fdcf-4e90-bca7-7ecbe1c407c1\") " pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:10.773543 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.773523 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfd9c\" (UniqueName: \"kubernetes.io/projected/69986027-fdcf-4e90-bca7-7ecbe1c407c1-kube-api-access-lfd9c\") pod \"authorino-5d4b67c446-sdnfm\" (UID: \"69986027-fdcf-4e90-bca7-7ecbe1c407c1\") " pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:10.865592 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.865559 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkpvv\" (UniqueName: \"kubernetes.io/projected/a05d9b65-5fe9-4354-884e-4c0e51c1b74f-kube-api-access-tkpvv\") pod \"authorino-58668d47d9-clmd2\" (UID: \"a05d9b65-5fe9-4354-884e-4c0e51c1b74f\") " pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:10.895760 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.895716 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58668d47d9-clmd2"] Jun 09 12:09:10.895935 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:10.895918 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tkpvv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-58668d47d9-clmd2" podUID="a05d9b65-5fe9-4354-884e-4c0e51c1b74f" Jun 09 12:09:10.925941 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.925874 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86bcc5b785-2dxxs"] Jun 09 12:09:10.932567 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.932541 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:10.935677 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.935656 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 09 12:09:10.940164 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.940144 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86bcc5b785-2dxxs"] Jun 09 12:09:10.966087 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.966067 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkpvv\" (UniqueName: \"kubernetes.io/projected/a05d9b65-5fe9-4354-884e-4c0e51c1b74f-kube-api-access-tkpvv\") pod \"authorino-58668d47d9-clmd2\" (UID: \"a05d9b65-5fe9-4354-884e-4c0e51c1b74f\") " pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:10.974264 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:10.974238 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkpvv\" (UniqueName: \"kubernetes.io/projected/a05d9b65-5fe9-4354-884e-4c0e51c1b74f-kube-api-access-tkpvv\") pod \"authorino-58668d47d9-clmd2\" (UID: \"a05d9b65-5fe9-4354-884e-4c0e51c1b74f\") " pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:11.067180 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.067131 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqk84\" (UniqueName: \"kubernetes.io/projected/5611b868-418b-40dd-9f50-5be34447ef6f-kube-api-access-tqk84\") pod \"authorino-86bcc5b785-2dxxs\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.067329 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.067195 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5611b868-418b-40dd-9f50-5be34447ef6f-tls-cert\") pod \"authorino-86bcc5b785-2dxxs\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.167791 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.167758 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqk84\" (UniqueName: \"kubernetes.io/projected/5611b868-418b-40dd-9f50-5be34447ef6f-kube-api-access-tqk84\") pod \"authorino-86bcc5b785-2dxxs\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.167897 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.167798 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5611b868-418b-40dd-9f50-5be34447ef6f-tls-cert\") pod \"authorino-86bcc5b785-2dxxs\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.170100 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.170075 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5611b868-418b-40dd-9f50-5be34447ef6f-tls-cert\") pod \"authorino-86bcc5b785-2dxxs\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.179949 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.179898 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqk84\" (UniqueName: \"kubernetes.io/projected/5611b868-418b-40dd-9f50-5be34447ef6f-kube-api-access-tqk84\") pod \"authorino-86bcc5b785-2dxxs\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.242653 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.242633 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:09:11.283235 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.283212 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:11.283366 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.283212 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:11.291124 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.291103 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:11.295960 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.295938 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:11.365354 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.365322 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86bcc5b785-2dxxs"] Jun 09 12:09:11.368612 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.368584 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkpvv\" (UniqueName: \"kubernetes.io/projected/a05d9b65-5fe9-4354-884e-4c0e51c1b74f-kube-api-access-tkpvv\") pod \"a05d9b65-5fe9-4354-884e-4c0e51c1b74f\" (UID: \"a05d9b65-5fe9-4354-884e-4c0e51c1b74f\") " Jun 09 12:09:11.368709 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.368666 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfd9c\" (UniqueName: \"kubernetes.io/projected/69986027-fdcf-4e90-bca7-7ecbe1c407c1-kube-api-access-lfd9c\") pod \"69986027-fdcf-4e90-bca7-7ecbe1c407c1\" (UID: \"69986027-fdcf-4e90-bca7-7ecbe1c407c1\") " Jun 09 12:09:11.369609 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:09:11.369571 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5611b868_418b_40dd_9f50_5be34447ef6f.slice/crio-895015ca3a9dc1ae517b37ff9feb7597a8498a199bc93a9990ca041f7a9dc616 WatchSource:0}: Error finding container 895015ca3a9dc1ae517b37ff9feb7597a8498a199bc93a9990ca041f7a9dc616: Status 404 returned error can't find the container with id 895015ca3a9dc1ae517b37ff9feb7597a8498a199bc93a9990ca041f7a9dc616 Jun 09 12:09:11.370970 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.370943 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69986027-fdcf-4e90-bca7-7ecbe1c407c1-kube-api-access-lfd9c" (OuterVolumeSpecName: "kube-api-access-lfd9c") pod "69986027-fdcf-4e90-bca7-7ecbe1c407c1" (UID: "69986027-fdcf-4e90-bca7-7ecbe1c407c1"). InnerVolumeSpecName "kube-api-access-lfd9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:09:11.371052 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.370995 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a05d9b65-5fe9-4354-884e-4c0e51c1b74f-kube-api-access-tkpvv" (OuterVolumeSpecName: "kube-api-access-tkpvv") pod "a05d9b65-5fe9-4354-884e-4c0e51c1b74f" (UID: "a05d9b65-5fe9-4354-884e-4c0e51c1b74f"). InnerVolumeSpecName "kube-api-access-tkpvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:09:11.469813 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.469731 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfd9c\" (UniqueName: \"kubernetes.io/projected/69986027-fdcf-4e90-bca7-7ecbe1c407c1-kube-api-access-lfd9c\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:09:11.469813 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:11.469774 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkpvv\" (UniqueName: \"kubernetes.io/projected/a05d9b65-5fe9-4354-884e-4c0e51c1b74f-kube-api-access-tkpvv\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:09:12.288863 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.288814 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" event={"ID":"5611b868-418b-40dd-9f50-5be34447ef6f","Type":"ContainerStarted","Data":"48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b"} Jun 09 12:09:12.289292 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.288869 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58668d47d9-clmd2" Jun 09 12:09:12.289292 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.288885 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sdnfm" Jun 09 12:09:12.289292 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.288869 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" event={"ID":"5611b868-418b-40dd-9f50-5be34447ef6f","Type":"ContainerStarted","Data":"895015ca3a9dc1ae517b37ff9feb7597a8498a199bc93a9990ca041f7a9dc616"} Jun 09 12:09:12.304841 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.304795 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" podStartSLOduration=2.002606291 podStartE2EDuration="2.304781162s" totalImagesPullingTime="302.174871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:09:12.303645773 +0000 UTC m=+861.585462997" watchObservedRunningTime="2026-06-09 12:09:12.304781162 +0000 UTC m=+861.586598406" Jun 09 12:09:12.327628 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.327607 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58668d47d9-clmd2"] Jun 09 12:09:12.333263 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.333239 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kltn8"] Jun 09 12:09:12.333419 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.333398 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kltn8" podUID="5cd95749-5690-49a9-8d2f-1ec1a536bee8" containerName="authorino" containerID="cri-o://7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2" gracePeriod=30 Jun 09 12:09:12.336506 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.336419 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58668d47d9-clmd2"] Jun 09 12:09:12.358919 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.358897 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdnfm"] Jun 09 12:09:12.363524 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.363504 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sdnfm"] Jun 09 12:09:12.577899 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.577874 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:09:12.681703 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.681662 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krhcg\" (UniqueName: \"kubernetes.io/projected/5cd95749-5690-49a9-8d2f-1ec1a536bee8-kube-api-access-krhcg\") pod \"5cd95749-5690-49a9-8d2f-1ec1a536bee8\" (UID: \"5cd95749-5690-49a9-8d2f-1ec1a536bee8\") " Jun 09 12:09:12.683727 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.683705 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cd95749-5690-49a9-8d2f-1ec1a536bee8-kube-api-access-krhcg" (OuterVolumeSpecName: "kube-api-access-krhcg") pod "5cd95749-5690-49a9-8d2f-1ec1a536bee8" (UID: "5cd95749-5690-49a9-8d2f-1ec1a536bee8"). InnerVolumeSpecName "kube-api-access-krhcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:09:12.782450 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:12.782419 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krhcg\" (UniqueName: \"kubernetes.io/projected/5cd95749-5690-49a9-8d2f-1ec1a536bee8-kube-api-access-krhcg\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:09:13.200425 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.200393 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69986027-fdcf-4e90-bca7-7ecbe1c407c1" path="/var/lib/kubelet/pods/69986027-fdcf-4e90-bca7-7ecbe1c407c1/volumes" Jun 09 12:09:13.200678 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.200661 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a05d9b65-5fe9-4354-884e-4c0e51c1b74f" path="/var/lib/kubelet/pods/a05d9b65-5fe9-4354-884e-4c0e51c1b74f/volumes" Jun 09 12:09:13.293321 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.293287 2550 generic.go:356] "Generic (PLEG): container finished" podID="5cd95749-5690-49a9-8d2f-1ec1a536bee8" containerID="7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2" exitCode=0 Jun 09 12:09:13.293778 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.293337 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kltn8" Jun 09 12:09:13.293778 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.293365 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kltn8" event={"ID":"5cd95749-5690-49a9-8d2f-1ec1a536bee8","Type":"ContainerDied","Data":"7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2"} Jun 09 12:09:13.293778 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.293416 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kltn8" event={"ID":"5cd95749-5690-49a9-8d2f-1ec1a536bee8","Type":"ContainerDied","Data":"e3b92d27055df549f5ab24a3284ab9ba48c53052715c1c91a7d0707628759cd8"} Jun 09 12:09:13.293778 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.293442 2550 scope.go:117] "RemoveContainer" containerID="7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2" Jun 09 12:09:13.301329 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.301313 2550 scope.go:117] "RemoveContainer" containerID="7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2" Jun 09 12:09:13.301575 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:13.301557 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2\": container with ID starting with 7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2 not found: ID does not exist" containerID="7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2" Jun 09 12:09:13.301629 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.301586 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2"} err="failed to get container status \"7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2\": rpc error: code = NotFound desc = could not find container \"7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2\": container with ID starting with 7d046114c46f979646bb1c16ee3f2bf78fb99537c9b501192628ed539fd789f2 not found: ID does not exist" Jun 09 12:09:13.307453 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.307433 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kltn8"] Jun 09 12:09:13.311509 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:13.311487 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kltn8"] Jun 09 12:09:15.200359 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.200316 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cd95749-5690-49a9-8d2f-1ec1a536bee8" path="/var/lib/kubelet/pods/5cd95749-5690-49a9-8d2f-1ec1a536bee8/volumes" Jun 09 12:09:15.811970 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.811936 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:15.828785 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.828753 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-645b688d57-5brk8"] Jun 09 12:09:15.829016 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.829004 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cd95749-5690-49a9-8d2f-1ec1a536bee8" containerName="authorino" Jun 09 12:09:15.829060 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.829017 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cd95749-5690-49a9-8d2f-1ec1a536bee8" containerName="authorino" Jun 09 12:09:15.829094 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.829072 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cd95749-5690-49a9-8d2f-1ec1a536bee8" containerName="authorino" Jun 09 12:09:15.831937 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.831922 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:15.834239 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.834221 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 12:09:15.835637 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.835619 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mz24z\"" Jun 09 12:09:15.835982 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.835964 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 12:09:15.847167 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.847145 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645b688d57-5brk8"] Jun 09 12:09:15.888565 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.888543 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:15.981429 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.981402 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869b98d549-2nv64"] Jun 09 12:09:15.984424 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.984409 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:15.996852 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:15.996824 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869b98d549-2nv64"] Jun 09 12:09:16.005802 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.005778 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcqch\" (UniqueName: \"kubernetes.io/projected/0606414d-2e0c-4a90-a9de-4254716f8c12-kube-api-access-fcqch\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.005915 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.005825 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.106856 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.106790 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rknnf\" (UniqueName: \"kubernetes.io/projected/b6a8d293-a6bf-4b62-a915-17a536bc3593-kube-api-access-rknnf\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.106856 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.106824 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcqch\" (UniqueName: \"kubernetes.io/projected/0606414d-2e0c-4a90-a9de-4254716f8c12-kube-api-access-fcqch\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.107027 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.106856 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.107027 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.106993 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.107123 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:16.107070 2550 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 12:09:16.107165 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:16.107134 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert podName:0606414d-2e0c-4a90-a9de-4254716f8c12 nodeName:}" failed. No retries permitted until 2026-06-09 12:09:16.607113089 +0000 UTC m=+865.888930301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert") pod "maas-controller-645b688d57-5brk8" (UID: "0606414d-2e0c-4a90-a9de-4254716f8c12") : secret "maas-controller-webhook-cert" not found Jun 09 12:09:16.117608 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.117579 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcqch\" (UniqueName: \"kubernetes.io/projected/0606414d-2e0c-4a90-a9de-4254716f8c12-kube-api-access-fcqch\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.207951 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.207923 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.208347 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.208029 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rknnf\" (UniqueName: \"kubernetes.io/projected/b6a8d293-a6bf-4b62-a915-17a536bc3593-kube-api-access-rknnf\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.208347 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:16.208070 2550 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 12:09:16.208347 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:16.208132 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert podName:b6a8d293-a6bf-4b62-a915-17a536bc3593 nodeName:}" failed. No retries permitted until 2026-06-09 12:09:16.708118818 +0000 UTC m=+865.989936018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert") pod "maas-controller-869b98d549-2nv64" (UID: "b6a8d293-a6bf-4b62-a915-17a536bc3593") : secret "maas-controller-webhook-cert" not found Jun 09 12:09:16.216968 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.216942 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rknnf\" (UniqueName: \"kubernetes.io/projected/b6a8d293-a6bf-4b62-a915-17a536bc3593-kube-api-access-rknnf\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.353175 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.353145 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:16.610521 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.610482 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.612863 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.612844 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert\") pod \"maas-controller-645b688d57-5brk8\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.711787 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.711763 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.714412 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.714387 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert\") pod \"maas-controller-869b98d549-2nv64\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:16.741983 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.741951 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:16.894199 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:16.894169 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:17.017972 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:17.017946 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869b98d549-2nv64"] Jun 09 12:09:17.020233 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:09:17.020199 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a8d293_a6bf_4b62_a915_17a536bc3593.slice/crio-a9e7b0767ef957e60499d489db9d8c657298d576b494ca5dd1a1db1354896155 WatchSource:0}: Error finding container a9e7b0767ef957e60499d489db9d8c657298d576b494ca5dd1a1db1354896155: Status 404 returned error can't find the container with id a9e7b0767ef957e60499d489db9d8c657298d576b494ca5dd1a1db1354896155 Jun 09 12:09:17.099198 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:17.099175 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645b688d57-5brk8"] Jun 09 12:09:17.101588 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:09:17.101565 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0606414d_2e0c_4a90_a9de_4254716f8c12.slice/crio-378516d323ffbde4009b303e39b26b493b5304e42da1566dbf2238b57b620318 WatchSource:0}: Error finding container 378516d323ffbde4009b303e39b26b493b5304e42da1566dbf2238b57b620318: Status 404 returned error can't find the container with id 378516d323ffbde4009b303e39b26b493b5304e42da1566dbf2238b57b620318 Jun 09 12:09:17.311140 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:17.311065 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869b98d549-2nv64" event={"ID":"b6a8d293-a6bf-4b62-a915-17a536bc3593","Type":"ContainerStarted","Data":"a9e7b0767ef957e60499d489db9d8c657298d576b494ca5dd1a1db1354896155"} Jun 09 12:09:17.312111 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:17.312090 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645b688d57-5brk8" event={"ID":"0606414d-2e0c-4a90-a9de-4254716f8c12","Type":"ContainerStarted","Data":"378516d323ffbde4009b303e39b26b493b5304e42da1566dbf2238b57b620318"} Jun 09 12:09:19.985133 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:19.985055 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:20.087585 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.087544 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:20.327926 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.327833 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869b98d549-2nv64" event={"ID":"b6a8d293-a6bf-4b62-a915-17a536bc3593","Type":"ContainerStarted","Data":"aaf2dc6d9fc5b838523417c310d0e555464e6cdec87556fde47ed12801edd834"} Jun 09 12:09:20.328092 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.328012 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:20.329466 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.329434 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645b688d57-5brk8" event={"ID":"0606414d-2e0c-4a90-a9de-4254716f8c12","Type":"ContainerStarted","Data":"9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52"} Jun 09 12:09:20.329567 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.329493 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:20.347781 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.347715 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-869b98d549-2nv64" podStartSLOduration=2.638772671 podStartE2EDuration="5.34770104s" totalImagesPullingTime="2.708928369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:09:20.346429124 +0000 UTC m=+869.628246343" watchObservedRunningTime="2026-06-09 12:09:20.34770104 +0000 UTC m=+869.629518262" Jun 09 12:09:20.363221 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:20.363183 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-645b688d57-5brk8" podStartSLOduration=2.7267866 podStartE2EDuration="5.36316987s" totalImagesPullingTime="2.63638327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:09:20.362571682 +0000 UTC m=+869.644388906" watchObservedRunningTime="2026-06-09 12:09:20.36316987 +0000 UTC m=+869.644987092" Jun 09 12:09:31.339798 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.339765 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:31.340233 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.339841 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:09:31.467926 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.467888 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645b688d57-5brk8"] Jun 09 12:09:31.468197 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.468172 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-645b688d57-5brk8" podUID="0606414d-2e0c-4a90-a9de-4254716f8c12" containerName="manager" containerID="cri-o://9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52" gracePeriod=30 Jun 09 12:09:31.703183 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.703160 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:31.828922 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.828885 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcqch\" (UniqueName: \"kubernetes.io/projected/0606414d-2e0c-4a90-a9de-4254716f8c12-kube-api-access-fcqch\") pod \"0606414d-2e0c-4a90-a9de-4254716f8c12\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " Jun 09 12:09:31.829081 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.828961 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert\") pod \"0606414d-2e0c-4a90-a9de-4254716f8c12\" (UID: \"0606414d-2e0c-4a90-a9de-4254716f8c12\") " Jun 09 12:09:31.831015 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.830980 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0606414d-2e0c-4a90-a9de-4254716f8c12" (UID: "0606414d-2e0c-4a90-a9de-4254716f8c12"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 12:09:31.831122 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.831017 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0606414d-2e0c-4a90-a9de-4254716f8c12-kube-api-access-fcqch" (OuterVolumeSpecName: "kube-api-access-fcqch") pod "0606414d-2e0c-4a90-a9de-4254716f8c12" (UID: "0606414d-2e0c-4a90-a9de-4254716f8c12"). InnerVolumeSpecName "kube-api-access-fcqch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:09:31.930517 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.930411 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcqch\" (UniqueName: \"kubernetes.io/projected/0606414d-2e0c-4a90-a9de-4254716f8c12-kube-api-access-fcqch\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:09:31.930517 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:31.930460 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0606414d-2e0c-4a90-a9de-4254716f8c12-webhook-cert\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:09:32.382834 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.382801 2550 generic.go:356] "Generic (PLEG): container finished" podID="0606414d-2e0c-4a90-a9de-4254716f8c12" containerID="9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52" exitCode=0 Jun 09 12:09:32.383304 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.382841 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645b688d57-5brk8" event={"ID":"0606414d-2e0c-4a90-a9de-4254716f8c12","Type":"ContainerDied","Data":"9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52"} Jun 09 12:09:32.383304 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.382846 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645b688d57-5brk8" Jun 09 12:09:32.383304 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.382882 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645b688d57-5brk8" event={"ID":"0606414d-2e0c-4a90-a9de-4254716f8c12","Type":"ContainerDied","Data":"378516d323ffbde4009b303e39b26b493b5304e42da1566dbf2238b57b620318"} Jun 09 12:09:32.383304 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.382905 2550 scope.go:117] "RemoveContainer" containerID="9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52" Jun 09 12:09:32.391086 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.391068 2550 scope.go:117] "RemoveContainer" containerID="9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52" Jun 09 12:09:32.391325 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:09:32.391306 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52\": container with ID starting with 9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52 not found: ID does not exist" containerID="9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52" Jun 09 12:09:32.391379 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.391336 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52"} err="failed to get container status \"9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52\": rpc error: code = NotFound desc = could not find container \"9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52\": container with ID starting with 9fe8e515ca4ad0d42bf5bb3b2c2693b87996ee66a825122a3a808f3b98746a52 not found: ID does not exist" Jun 09 12:09:32.405974 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.405946 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-645b688d57-5brk8"] Jun 09 12:09:32.413391 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:32.413372 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-645b688d57-5brk8"] Jun 09 12:09:33.200947 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:33.200909 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0606414d-2e0c-4a90-a9de-4254716f8c12" path="/var/lib/kubelet/pods/0606414d-2e0c-4a90-a9de-4254716f8c12/volumes" Jun 09 12:09:37.429265 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:37.429230 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:50.986190 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:50.986149 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:51.041752 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.041708 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-796d7c9c54-548kx"] Jun 09 12:09:51.042042 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.042028 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0606414d-2e0c-4a90-a9de-4254716f8c12" containerName="manager" Jun 09 12:09:51.042084 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.042046 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0606414d-2e0c-4a90-a9de-4254716f8c12" containerName="manager" Jun 09 12:09:51.042159 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.042117 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0606414d-2e0c-4a90-a9de-4254716f8c12" containerName="manager" Jun 09 12:09:51.045171 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.045155 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.047837 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.047809 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-xkvhg\"" Jun 09 12:09:51.047971 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.047953 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 09 12:09:51.059533 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.059506 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-796d7c9c54-548kx"] Jun 09 12:09:51.059873 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.059854 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nr5\" (UniqueName: \"kubernetes.io/projected/8b32e524-d3e9-4463-ae37-f14e94aa655e-kube-api-access-g9nr5\") pod \"maas-api-796d7c9c54-548kx\" (UID: \"8b32e524-d3e9-4463-ae37-f14e94aa655e\") " pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.059973 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.059890 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b32e524-d3e9-4463-ae37-f14e94aa655e-maas-api-tls\") pod \"maas-api-796d7c9c54-548kx\" (UID: \"8b32e524-d3e9-4463-ae37-f14e94aa655e\") " pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.160234 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.160205 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nr5\" (UniqueName: \"kubernetes.io/projected/8b32e524-d3e9-4463-ae37-f14e94aa655e-kube-api-access-g9nr5\") pod \"maas-api-796d7c9c54-548kx\" (UID: \"8b32e524-d3e9-4463-ae37-f14e94aa655e\") " pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.160385 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.160240 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b32e524-d3e9-4463-ae37-f14e94aa655e-maas-api-tls\") pod \"maas-api-796d7c9c54-548kx\" (UID: \"8b32e524-d3e9-4463-ae37-f14e94aa655e\") " pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.162679 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.162655 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 09 12:09:51.168976 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.168951 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nr5\" (UniqueName: \"kubernetes.io/projected/8b32e524-d3e9-4463-ae37-f14e94aa655e-kube-api-access-g9nr5\") pod \"maas-api-796d7c9c54-548kx\" (UID: \"8b32e524-d3e9-4463-ae37-f14e94aa655e\") " pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.172542 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.172521 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b32e524-d3e9-4463-ae37-f14e94aa655e-maas-api-tls\") pod \"maas-api-796d7c9c54-548kx\" (UID: \"8b32e524-d3e9-4463-ae37-f14e94aa655e\") " pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.206935 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.206912 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:51.357095 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.357031 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-xkvhg\"" Jun 09 12:09:51.365225 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.365209 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:51.443784 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.443755 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:51.503876 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.503808 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-796d7c9c54-548kx"] Jun 09 12:09:51.506041 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:09:51.506010 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b32e524_d3e9_4463_ae37_f14e94aa655e.slice/crio-88d836531a38db4b4757dfe45c218d4e817dd659b22f4882edae483ce171e444 WatchSource:0}: Error finding container 88d836531a38db4b4757dfe45c218d4e817dd659b22f4882edae483ce171e444: Status 404 returned error can't find the container with id 88d836531a38db4b4757dfe45c218d4e817dd659b22f4882edae483ce171e444 Jun 09 12:09:51.509085 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.509056 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:51.545787 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.545756 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-2r85z"] Jun 09 12:09:51.550331 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.550313 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:09:51.552953 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.552930 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ss9ft\"" Jun 09 12:09:51.552953 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.552944 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 09 12:09:51.558138 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.558118 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-2r85z"] Jun 09 12:09:51.561146 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.561125 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-7mmmh"] Jun 09 12:09:51.564063 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.564045 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:09:51.564234 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.564214 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvfsf\" (UniqueName: \"kubernetes.io/projected/6cd261fb-a5cd-4205-87d8-07100f9c227d-kube-api-access-kvfsf\") pod \"payload-pre-processing-67d7689b65-2r85z\" (UID: \"6cd261fb-a5cd-4205-87d8-07100f9c227d\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:09:51.575140 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.575117 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-7mmmh"] Jun 09 12:09:51.665473 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.665440 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvfsf\" (UniqueName: \"kubernetes.io/projected/6cd261fb-a5cd-4205-87d8-07100f9c227d-kube-api-access-kvfsf\") pod \"payload-pre-processing-67d7689b65-2r85z\" (UID: \"6cd261fb-a5cd-4205-87d8-07100f9c227d\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:09:51.665646 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.665504 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8s55\" (UniqueName: \"kubernetes.io/projected/f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c-kube-api-access-w8s55\") pod \"payload-processing-7c6c859784-7mmmh\" (UID: \"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c\") " pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:09:51.689055 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.689028 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvfsf\" (UniqueName: \"kubernetes.io/projected/6cd261fb-a5cd-4205-87d8-07100f9c227d-kube-api-access-kvfsf\") pod \"payload-pre-processing-67d7689b65-2r85z\" (UID: \"6cd261fb-a5cd-4205-87d8-07100f9c227d\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:09:51.699485 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.699459 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:51.766515 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.766483 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8s55\" (UniqueName: \"kubernetes.io/projected/f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c-kube-api-access-w8s55\") pod \"payload-processing-7c6c859784-7mmmh\" (UID: \"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c\") " pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:09:51.778885 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.778858 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8s55\" (UniqueName: \"kubernetes.io/projected/f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c-kube-api-access-w8s55\") pod \"payload-processing-7c6c859784-7mmmh\" (UID: \"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c\") " pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:09:51.796904 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.796871 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:09:51.862025 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.861999 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:09:51.873166 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:51.873146 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:09:52.001663 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:52.001605 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-2r85z"] Jun 09 12:09:52.004255 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:09:52.004225 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd261fb_a5cd_4205_87d8_07100f9c227d.slice/crio-7984e90339b14382cd61b0cacddb397b7d0081d799b77c0fa50224357c3f8be8 WatchSource:0}: Error finding container 7984e90339b14382cd61b0cacddb397b7d0081d799b77c0fa50224357c3f8be8: Status 404 returned error can't find the container with id 7984e90339b14382cd61b0cacddb397b7d0081d799b77c0fa50224357c3f8be8 Jun 09 12:09:52.031057 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:52.031036 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-7mmmh"] Jun 09 12:09:52.033289 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:09:52.033264 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c66d2a_e7e3_4ec3_9ba2_dffa7429803c.slice/crio-6bf384bfdff1a9c85a66c5cfb216ac4a398b763aca25a746ec48ba20570158c5 WatchSource:0}: Error finding container 6bf384bfdff1a9c85a66c5cfb216ac4a398b763aca25a746ec48ba20570158c5: Status 404 returned error can't find the container with id 6bf384bfdff1a9c85a66c5cfb216ac4a398b763aca25a746ec48ba20570158c5 Jun 09 12:09:52.462661 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:52.462622 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerStarted","Data":"6bf384bfdff1a9c85a66c5cfb216ac4a398b763aca25a746ec48ba20570158c5"} Jun 09 12:09:52.463678 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:52.463651 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" event={"ID":"6cd261fb-a5cd-4205-87d8-07100f9c227d","Type":"ContainerStarted","Data":"7984e90339b14382cd61b0cacddb397b7d0081d799b77c0fa50224357c3f8be8"} Jun 09 12:09:52.464643 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:52.464620 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-796d7c9c54-548kx" event={"ID":"8b32e524-d3e9-4463-ae37-f14e94aa655e","Type":"ContainerStarted","Data":"88d836531a38db4b4757dfe45c218d4e817dd659b22f4882edae483ce171e444"} Jun 09 12:09:54.474722 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.474685 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerStarted","Data":"e778ba597992ad60dfbd8cb600ae302b8c59ffe2a05bf0aa918c79dc61e5b861"} Jun 09 12:09:54.475227 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.474773 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:09:54.476183 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.476161 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" event={"ID":"6cd261fb-a5cd-4205-87d8-07100f9c227d","Type":"ContainerStarted","Data":"6cc4abee8004b58cdf7c423ebf6bd69d87b30eb1096a0a505212b503e0d344a4"} Jun 09 12:09:54.476315 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.476275 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:09:54.477569 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.477550 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-796d7c9c54-548kx" event={"ID":"8b32e524-d3e9-4463-ae37-f14e94aa655e","Type":"ContainerStarted","Data":"f517e99c5112ea19ec90575eedc2ac1f470294a04ffad723e00c05f741c68dce"} Jun 09 12:09:54.477689 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.477674 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:09:54.492971 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.492925 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podStartSLOduration=1.8684173670000002 podStartE2EDuration="3.492908333s" totalImagesPullingTime="1.624490966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:09:54.49156157 +0000 UTC m=+903.773378792" watchObservedRunningTime="2026-06-09 12:09:54.492908333 +0000 UTC m=+903.774725538" Jun 09 12:09:54.506236 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.506195 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" podStartSLOduration=1.850395342 podStartE2EDuration="3.506183996s" totalImagesPullingTime="1.655788654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:09:54.505978539 +0000 UTC m=+903.787795762" watchObservedRunningTime="2026-06-09 12:09:54.506183996 +0000 UTC m=+903.788001219" Jun 09 12:09:54.523197 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:09:54.523158 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-796d7c9c54-548kx" podStartSLOduration=1.370768858 podStartE2EDuration="3.523145247s" totalImagesPullingTime="2.152376389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:09:54.522033123 +0000 UTC m=+903.803850346" watchObservedRunningTime="2026-06-09 12:09:54.523145247 +0000 UTC m=+903.804962470" Jun 09 12:10:00.487655 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:00.487621 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-796d7c9c54-548kx" Jun 09 12:10:02.257459 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:02.257424 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:10:02.257858 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:02.257592 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" containerID="cri-o://17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459" gracePeriod=30 Jun 09 12:10:04.294395 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.294368 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.361152 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.361124 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxg6q\" (UniqueName: \"kubernetes.io/projected/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16-kube-api-access-pxg6q\") pod \"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16\" (UID: \"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16\") " Jun 09 12:10:04.363280 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.363252 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16-kube-api-access-pxg6q" (OuterVolumeSpecName: "kube-api-access-pxg6q") pod "ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" (UID: "ab25a28a-aae4-4f7f-9ed3-df1faa6afb16"). InnerVolumeSpecName "kube-api-access-pxg6q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:10:04.462474 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.462408 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxg6q\" (UniqueName: \"kubernetes.io/projected/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16-kube-api-access-pxg6q\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:10:04.528381 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.528344 2550 generic.go:356] "Generic (PLEG): container finished" podID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerID="17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459" exitCode=143 Jun 09 12:10:04.528381 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.528383 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16","Type":"ContainerDied","Data":"17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459"} Jun 09 12:10:04.528830 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.528399 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.528830 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.528445 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"ab25a28a-aae4-4f7f-9ed3-df1faa6afb16","Type":"ContainerDied","Data":"3456fd79daca692b461a717121106328e99de9c0414eb8274b3881bbbf7d77da"} Jun 09 12:10:04.528830 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.528465 2550 scope.go:117] "RemoveContainer" containerID="17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459" Jun 09 12:10:04.539042 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.538944 2550 scope.go:117] "RemoveContainer" containerID="17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459" Jun 09 12:10:04.539202 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:10:04.539184 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459\": container with ID starting with 17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459 not found: ID does not exist" containerID="17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459" Jun 09 12:10:04.539257 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.539209 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459"} err="failed to get container status \"17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459\": rpc error: code = NotFound desc = could not find container \"17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459\": container with ID starting with 17f08146844ee68a867eb73bd76b01c063416204c639821e0ea5a34b4ad43459 not found: ID does not exist" Jun 09 12:10:04.553208 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.553188 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:10:04.556957 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.556937 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:10:04.585041 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.585022 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:10:04.585290 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.585277 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" Jun 09 12:10:04.585333 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.585290 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" Jun 09 12:10:04.585372 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.585345 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" containerName="keycloak" Jun 09 12:10:04.589130 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.589115 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.592101 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.592083 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 12:10:04.592416 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.592391 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 12:10:04.592555 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.592395 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-wq76x\"" Jun 09 12:10:04.592555 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.592468 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 09 12:10:04.593126 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.593106 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 12:10:04.601580 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.601560 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:10:04.663720 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.663691 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sgzx\" (UniqueName: \"kubernetes.io/projected/f29cfb5b-dac5-4d81-b0cb-c39bd84e7164-kube-api-access-7sgzx\") pod \"maas-keycloak-0\" (UID: \"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.663875 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.663812 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f29cfb5b-dac5-4d81-b0cb-c39bd84e7164-test-realms\") pod \"maas-keycloak-0\" (UID: \"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.764497 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.764410 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f29cfb5b-dac5-4d81-b0cb-c39bd84e7164-test-realms\") pod \"maas-keycloak-0\" (UID: \"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.764497 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.764465 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sgzx\" (UniqueName: \"kubernetes.io/projected/f29cfb5b-dac5-4d81-b0cb-c39bd84e7164-kube-api-access-7sgzx\") pod \"maas-keycloak-0\" (UID: \"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.765244 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.765221 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f29cfb5b-dac5-4d81-b0cb-c39bd84e7164-test-realms\") pod \"maas-keycloak-0\" (UID: \"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.781896 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.781866 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sgzx\" (UniqueName: \"kubernetes.io/projected/f29cfb5b-dac5-4d81-b0cb-c39bd84e7164-kube-api-access-7sgzx\") pod \"maas-keycloak-0\" (UID: \"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164\") " pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:04.898114 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:04.898080 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:05.040403 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.040378 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 12:10:05.043015 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:10:05.042984 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29cfb5b_dac5_4d81_b0cb_c39bd84e7164.slice/crio-621d857174934b4e121d33ae2bb12e51f533375eb8e5dca8b50879e000cd0e35 WatchSource:0}: Error finding container 621d857174934b4e121d33ae2bb12e51f533375eb8e5dca8b50879e000cd0e35: Status 404 returned error can't find the container with id 621d857174934b4e121d33ae2bb12e51f533375eb8e5dca8b50879e000cd0e35 Jun 09 12:10:05.201503 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.201470 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab25a28a-aae4-4f7f-9ed3-df1faa6afb16" path="/var/lib/kubelet/pods/ab25a28a-aae4-4f7f-9ed3-df1faa6afb16/volumes" Jun 09 12:10:05.484586 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.484558 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-2r85z" Jun 09 12:10:05.485052 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.484644 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:10:05.534400 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.534363 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164","Type":"ContainerStarted","Data":"04b8f855727abcd6d6067a609a1e1de110c965d78605d8ecccc2995747866e34"} Jun 09 12:10:05.534586 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.534432 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f29cfb5b-dac5-4d81-b0cb-c39bd84e7164","Type":"ContainerStarted","Data":"621d857174934b4e121d33ae2bb12e51f533375eb8e5dca8b50879e000cd0e35"} Jun 09 12:10:05.554947 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.554898 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.222978162 podStartE2EDuration="1.554875108s" totalImagesPullingTime="331.896946ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:10:05.554107248 +0000 UTC m=+914.835924472" watchObservedRunningTime="2026-06-09 12:10:05.554875108 +0000 UTC m=+914.836692332" Jun 09 12:10:05.899012 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.898972 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:05.900634 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:05.900593 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:06.899149 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:06.899099 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:07.899407 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:07.899357 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:08.899284 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:08.899221 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:09.898939 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:09.898886 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:10.898817 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:10.898761 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:11.899075 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:11.899022 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:12.898829 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:12.898775 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:13.899250 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:13.899196 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:14.898417 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:14.898381 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:14.898668 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:14.898631 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:15.899166 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:15.899116 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:16.899397 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:16.899343 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:17.898499 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:17.898445 2550 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 09 12:10:18.990395 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:18.990355 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:19.004809 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:19.004760 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f29cfb5b-dac5-4d81-b0cb-c39bd84e7164" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 12:10:28.996825 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:28.996789 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 12:10:39.714852 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.714819 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-678947585c-hfkrn"] Jun 09 12:10:39.719504 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.719484 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.722092 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.722071 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 09 12:10:39.727823 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.727802 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678947585c-hfkrn"] Jun 09 12:10:39.876322 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.876288 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c68ffdf-2617-4f89-84ef-b663b658de3d-tls-cert\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.876489 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.876343 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0c68ffdf-2617-4f89-84ef-b663b658de3d-oidc-ca\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.876489 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.876397 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfgvc\" (UniqueName: \"kubernetes.io/projected/0c68ffdf-2617-4f89-84ef-b663b658de3d-kube-api-access-rfgvc\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.977872 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.977784 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0c68ffdf-2617-4f89-84ef-b663b658de3d-oidc-ca\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.977872 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.977821 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfgvc\" (UniqueName: \"kubernetes.io/projected/0c68ffdf-2617-4f89-84ef-b663b658de3d-kube-api-access-rfgvc\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.977872 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.977868 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c68ffdf-2617-4f89-84ef-b663b658de3d-tls-cert\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.978444 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.978423 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0c68ffdf-2617-4f89-84ef-b663b658de3d-oidc-ca\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.980244 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.980224 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c68ffdf-2617-4f89-84ef-b663b658de3d-tls-cert\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:39.987443 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:39.987416 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfgvc\" (UniqueName: \"kubernetes.io/projected/0c68ffdf-2617-4f89-84ef-b663b658de3d-kube-api-access-rfgvc\") pod \"authorino-678947585c-hfkrn\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:40.029233 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.029202 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:10:40.158688 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.158659 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678947585c-hfkrn"] Jun 09 12:10:40.161549 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:10:40.161519 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c68ffdf_2617_4f89_84ef_b663b658de3d.slice/crio-786378e028d1e3b249132caea02f1a01a053bef5844b444613c8150cdc724003 WatchSource:0}: Error finding container 786378e028d1e3b249132caea02f1a01a053bef5844b444613c8150cdc724003: Status 404 returned error can't find the container with id 786378e028d1e3b249132caea02f1a01a053bef5844b444613c8150cdc724003 Jun 09 12:10:40.713521 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.713481 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678947585c-hfkrn" event={"ID":"0c68ffdf-2617-4f89-84ef-b663b658de3d","Type":"ContainerStarted","Data":"883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f"} Jun 09 12:10:40.713675 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.713529 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678947585c-hfkrn" event={"ID":"0c68ffdf-2617-4f89-84ef-b663b658de3d","Type":"ContainerStarted","Data":"786378e028d1e3b249132caea02f1a01a053bef5844b444613c8150cdc724003"} Jun 09 12:10:40.732901 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.732840 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-678947585c-hfkrn" podStartSLOduration=1.290362626 podStartE2EDuration="1.732823613s" totalImagesPullingTime="442.460987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:10:40.732114372 +0000 UTC m=+950.013931596" watchObservedRunningTime="2026-06-09 12:10:40.732823613 +0000 UTC m=+950.014640836" Jun 09 12:10:40.768197 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.768161 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86bcc5b785-2dxxs"] Jun 09 12:10:40.768386 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:40.768347 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" podUID="5611b868-418b-40dd-9f50-5be34447ef6f" containerName="authorino" containerID="cri-o://48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b" gracePeriod=30 Jun 09 12:10:41.043019 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.042987 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:10:41.188119 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.188032 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqk84\" (UniqueName: \"kubernetes.io/projected/5611b868-418b-40dd-9f50-5be34447ef6f-kube-api-access-tqk84\") pod \"5611b868-418b-40dd-9f50-5be34447ef6f\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " Jun 09 12:10:41.188254 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.188121 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5611b868-418b-40dd-9f50-5be34447ef6f-tls-cert\") pod \"5611b868-418b-40dd-9f50-5be34447ef6f\" (UID: \"5611b868-418b-40dd-9f50-5be34447ef6f\") " Jun 09 12:10:41.191109 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.191072 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5611b868-418b-40dd-9f50-5be34447ef6f-kube-api-access-tqk84" (OuterVolumeSpecName: "kube-api-access-tqk84") pod "5611b868-418b-40dd-9f50-5be34447ef6f" (UID: "5611b868-418b-40dd-9f50-5be34447ef6f"). InnerVolumeSpecName "kube-api-access-tqk84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:10:41.198499 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.198469 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5611b868-418b-40dd-9f50-5be34447ef6f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5611b868-418b-40dd-9f50-5be34447ef6f" (UID: "5611b868-418b-40dd-9f50-5be34447ef6f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 12:10:41.215002 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.214977 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:10:41.289064 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.289017 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqk84\" (UniqueName: \"kubernetes.io/projected/5611b868-418b-40dd-9f50-5be34447ef6f-kube-api-access-tqk84\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:10:41.289064 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.289059 2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5611b868-418b-40dd-9f50-5be34447ef6f-tls-cert\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:10:41.718991 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.718955 2550 generic.go:356] "Generic (PLEG): container finished" podID="5611b868-418b-40dd-9f50-5be34447ef6f" containerID="48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b" exitCode=0 Jun 09 12:10:41.719177 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.718999 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" Jun 09 12:10:41.719177 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.719048 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" event={"ID":"5611b868-418b-40dd-9f50-5be34447ef6f","Type":"ContainerDied","Data":"48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b"} Jun 09 12:10:41.719177 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.719083 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86bcc5b785-2dxxs" event={"ID":"5611b868-418b-40dd-9f50-5be34447ef6f","Type":"ContainerDied","Data":"895015ca3a9dc1ae517b37ff9feb7597a8498a199bc93a9990ca041f7a9dc616"} Jun 09 12:10:41.719177 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.719102 2550 scope.go:117] "RemoveContainer" containerID="48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b" Jun 09 12:10:41.727581 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.727566 2550 scope.go:117] "RemoveContainer" containerID="48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b" Jun 09 12:10:41.727867 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:10:41.727848 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b\": container with ID starting with 48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b not found: ID does not exist" containerID="48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b" Jun 09 12:10:41.727913 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.727878 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b"} err="failed to get container status \"48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b\": rpc error: code = NotFound desc = could not find container \"48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b\": container with ID starting with 48d4fc912b2567e8c39e46e129256ff122b0b2112e30d83ab8875c5fa73e6b5b not found: ID does not exist" Jun 09 12:10:41.743065 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.743042 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86bcc5b785-2dxxs"] Jun 09 12:10:41.751279 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:41.751260 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86bcc5b785-2dxxs"] Jun 09 12:10:42.251730 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:42.251699 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:10:43.202256 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:43.202223 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5611b868-418b-40dd-9f50-5be34447ef6f" path="/var/lib/kubelet/pods/5611b868-418b-40dd-9f50-5be34447ef6f/volumes" Jun 09 12:10:48.913181 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:48.913146 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:10:52.380665 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.380635 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg"] Jun 09 12:10:52.381056 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.380936 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5611b868-418b-40dd-9f50-5be34447ef6f" containerName="authorino" Jun 09 12:10:52.381056 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.380948 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5611b868-418b-40dd-9f50-5be34447ef6f" containerName="authorino" Jun 09 12:10:52.381056 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.381010 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5611b868-418b-40dd-9f50-5be34447ef6f" containerName="authorino" Jun 09 12:10:52.385216 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.385200 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.387750 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.387720 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 12:10:52.388862 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.388836 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7lhz9\"" Jun 09 12:10:52.388862 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.388849 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 09 12:10:52.389031 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.388838 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 12:10:52.399378 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.399356 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg"] Jun 09 12:10:52.415034 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.415006 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:10:52.467424 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.467384 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.467596 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.467454 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335b52d-d0e1-4f7b-8901-99840e51fc82-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.467596 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.467478 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.467596 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.467502 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.467596 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.467534 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.467596 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.467559 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwtpm\" (UniqueName: \"kubernetes.io/projected/7335b52d-d0e1-4f7b-8901-99840e51fc82-kube-api-access-kwtpm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568101 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568065 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568101 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568101 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtpm\" (UniqueName: \"kubernetes.io/projected/7335b52d-d0e1-4f7b-8901-99840e51fc82-kube-api-access-kwtpm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568308 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568131 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568308 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568203 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335b52d-d0e1-4f7b-8901-99840e51fc82-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568308 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568238 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568308 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568280 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568494 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568473 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568599 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568570 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.568664 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.568622 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.570366 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.570345 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335b52d-d0e1-4f7b-8901-99840e51fc82-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.570637 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.570619 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335b52d-d0e1-4f7b-8901-99840e51fc82-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.575994 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.575971 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwtpm\" (UniqueName: \"kubernetes.io/projected/7335b52d-d0e1-4f7b-8901-99840e51fc82-kube-api-access-kwtpm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-ff4rg\" (UID: \"7335b52d-d0e1-4f7b-8901-99840e51fc82\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.695133 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.695055 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:10:52.825037 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:52.825004 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg"] Jun 09 12:10:52.828017 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:10:52.827987 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7335b52d_d0e1_4f7b_8901_99840e51fc82.slice/crio-b7ad2f41377ecc5a08a86f413ccf60b3d7f2383ab65e3519240afb627ded9023 WatchSource:0}: Error finding container b7ad2f41377ecc5a08a86f413ccf60b3d7f2383ab65e3519240afb627ded9023: Status 404 returned error can't find the container with id b7ad2f41377ecc5a08a86f413ccf60b3d7f2383ab65e3519240afb627ded9023 Jun 09 12:10:53.770620 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:53.770583 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" event={"ID":"7335b52d-d0e1-4f7b-8901-99840e51fc82","Type":"ContainerStarted","Data":"b7ad2f41377ecc5a08a86f413ccf60b3d7f2383ab65e3519240afb627ded9023"} Jun 09 12:10:56.783932 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:56.783895 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" event={"ID":"7335b52d-d0e1-4f7b-8901-99840e51fc82","Type":"ContainerStarted","Data":"1cb876727ee9de734aa2f9160dd7f8c3be9b5a6657287ac6f5e14c778b869142"} Jun 09 12:10:58.794305 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:58.794271 2550 generic.go:356] "Generic (PLEG): container finished" podID="7335b52d-d0e1-4f7b-8901-99840e51fc82" containerID="1cb876727ee9de734aa2f9160dd7f8c3be9b5a6657287ac6f5e14c778b869142" exitCode=0 Jun 09 12:10:58.794792 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:10:58.794350 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" event={"ID":"7335b52d-d0e1-4f7b-8901-99840e51fc82","Type":"ContainerDied","Data":"1cb876727ee9de734aa2f9160dd7f8c3be9b5a6657287ac6f5e14c778b869142"} Jun 09 12:11:00.805783 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:00.805723 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" event={"ID":"7335b52d-d0e1-4f7b-8901-99840e51fc82","Type":"ContainerStarted","Data":"401e4fa59a438ca8a18e4fdfd948aee3da89f04e56f8e331405a42997f761c2f"} Jun 09 12:11:00.827077 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:00.827013 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" podStartSLOduration=2.025727853 podStartE2EDuration="8.826997834s" totalImagesPullingTime="4.007844751s" totalInitContainerRuntime="2.79342523s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:10:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:11:00.825323196 +0000 UTC m=+970.107140435" watchObservedRunningTime="2026-06-09 12:11:00.826997834 +0000 UTC m=+970.108815056" Jun 09 12:11:02.695716 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:02.695680 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:11:02.696132 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:02.695756 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:11:02.708408 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:02.708381 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:11:02.824608 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:02.824587 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-ff4rg" Jun 09 12:11:20.024401 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:20.024324 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:11:22.175012 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.174979 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2"] Jun 09 12:11:22.189713 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.189672 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2"] Jun 09 12:11:22.189887 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.189802 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.192398 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.192377 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 09 12:11:22.261660 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.261585 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:11:22.287672 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.287643 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.287819 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.287682 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.287819 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.287782 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8781f4c2-468c-4a2f-b74d-99d546493f13-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.287906 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.287826 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrznr\" (UniqueName: \"kubernetes.io/projected/8781f4c2-468c-4a2f-b74d-99d546493f13-kube-api-access-jrznr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.287906 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.287861 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.287906 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.287893 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.388987 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.388952 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389117 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.388991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389117 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389043 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8781f4c2-468c-4a2f-b74d-99d546493f13-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389117 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389085 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrznr\" (UniqueName: \"kubernetes.io/projected/8781f4c2-468c-4a2f-b74d-99d546493f13-kube-api-access-jrznr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389117 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389114 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389324 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389143 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389537 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389510 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389654 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389540 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.389654 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.389586 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.391197 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.391178 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8781f4c2-468c-4a2f-b74d-99d546493f13-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.391471 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.391454 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8781f4c2-468c-4a2f-b74d-99d546493f13-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.396485 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.396464 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrznr\" (UniqueName: \"kubernetes.io/projected/8781f4c2-468c-4a2f-b74d-99d546493f13-kube-api-access-jrznr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2\" (UID: \"8781f4c2-468c-4a2f-b74d-99d546493f13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.500666 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.500597 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:11:22.627916 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.627887 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2"] Jun 09 12:11:22.629749 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:11:22.629707 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8781f4c2_468c_4a2f_b74d_99d546493f13.slice/crio-4003dd28e6f5e47cbc31c2a183dae12558345cd13f6c6a6fbf1ed06a752b8bd4 WatchSource:0}: Error finding container 4003dd28e6f5e47cbc31c2a183dae12558345cd13f6c6a6fbf1ed06a752b8bd4: Status 404 returned error can't find the container with id 4003dd28e6f5e47cbc31c2a183dae12558345cd13f6c6a6fbf1ed06a752b8bd4 Jun 09 12:11:22.896208 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.896172 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" event={"ID":"8781f4c2-468c-4a2f-b74d-99d546493f13","Type":"ContainerStarted","Data":"81381aa85526f908ab00bdef9bf53359fc7045085678b0fb4c8d46f27684d86c"} Jun 09 12:11:22.896393 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:22.896265 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" event={"ID":"8781f4c2-468c-4a2f-b74d-99d546493f13","Type":"ContainerStarted","Data":"4003dd28e6f5e47cbc31c2a183dae12558345cd13f6c6a6fbf1ed06a752b8bd4"} Jun 09 12:11:25.389719 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:25.389670 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:11:25.490361 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:25.490330 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:11:36.410024 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:36.409994 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:11:52.483135 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:52.483096 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:11:54.023012 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:54.022979 2550 generic.go:356] "Generic (PLEG): container finished" podID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" containerID="e778ba597992ad60dfbd8cb600ae302b8c59ffe2a05bf0aa918c79dc61e5b861" exitCode=1 Jun 09 12:11:54.023392 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:54.023036 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerDied","Data":"e778ba597992ad60dfbd8cb600ae302b8c59ffe2a05bf0aa918c79dc61e5b861"} Jun 09 12:11:54.023392 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:54.023317 2550 scope.go:117] "RemoveContainer" containerID="e778ba597992ad60dfbd8cb600ae302b8c59ffe2a05bf0aa918c79dc61e5b861" Jun 09 12:11:55.030215 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:55.030181 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerStarted","Data":"a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c"} Jun 09 12:11:55.030611 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:11:55.030390 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:12:06.035913 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:06.035881 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:12:56.668536 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.668502 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5df6655c5b-7lcbv"] Jun 09 12:12:56.672033 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.672010 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.679938 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.679917 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5df6655c5b-7lcbv"] Jun 09 12:12:56.809634 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.809599 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/db1e1072-5ade-445d-a904-cdf2214ecf69-tls-cert\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.809634 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.809631 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/db1e1072-5ade-445d-a904-cdf2214ecf69-oidc-ca\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.809895 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.809669 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phhk2\" (UniqueName: \"kubernetes.io/projected/db1e1072-5ade-445d-a904-cdf2214ecf69-kube-api-access-phhk2\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.910985 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.910937 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/db1e1072-5ade-445d-a904-cdf2214ecf69-tls-cert\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.910985 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.910989 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/db1e1072-5ade-445d-a904-cdf2214ecf69-oidc-ca\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.911216 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.911051 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phhk2\" (UniqueName: \"kubernetes.io/projected/db1e1072-5ade-445d-a904-cdf2214ecf69-kube-api-access-phhk2\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.911793 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.911775 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/db1e1072-5ade-445d-a904-cdf2214ecf69-oidc-ca\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.913405 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.913387 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/db1e1072-5ade-445d-a904-cdf2214ecf69-tls-cert\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.919432 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.919381 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phhk2\" (UniqueName: \"kubernetes.io/projected/db1e1072-5ade-445d-a904-cdf2214ecf69-kube-api-access-phhk2\") pod \"authorino-5df6655c5b-7lcbv\" (UID: \"db1e1072-5ade-445d-a904-cdf2214ecf69\") " pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:56.981825 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:56.981799 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5df6655c5b-7lcbv" Jun 09 12:12:57.113160 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:57.110900 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5df6655c5b-7lcbv"] Jun 09 12:12:57.114931 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:12:57.114903 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb1e1072_5ade_445d_a904_cdf2214ecf69.slice/crio-2545a3723b5449919e226a4a53d5d6a022b9e801ac2d36d3831dc7e34f6c3eb7 WatchSource:0}: Error finding container 2545a3723b5449919e226a4a53d5d6a022b9e801ac2d36d3831dc7e34f6c3eb7: Status 404 returned error can't find the container with id 2545a3723b5449919e226a4a53d5d6a022b9e801ac2d36d3831dc7e34f6c3eb7 Jun 09 12:12:57.116228 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:57.116208 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 12:12:57.273566 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:57.273487 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:12:57.291675 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:57.291633 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5df6655c5b-7lcbv" event={"ID":"db1e1072-5ade-445d-a904-cdf2214ecf69","Type":"ContainerStarted","Data":"2545a3723b5449919e226a4a53d5d6a022b9e801ac2d36d3831dc7e34f6c3eb7"} Jun 09 12:12:57.367910 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:57.367881 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:12:58.299290 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.299246 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5df6655c5b-7lcbv" event={"ID":"db1e1072-5ade-445d-a904-cdf2214ecf69","Type":"ContainerStarted","Data":"506a42d628d1b0a202bc6a699e6b1a03aaafffe899692aae94d099b9bf593708"} Jun 09 12:12:58.320722 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.320667 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5df6655c5b-7lcbv" podStartSLOduration=1.957615881 podStartE2EDuration="2.320649094s" totalImagesPullingTime="363.033213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:12:58.320243566 +0000 UTC m=+1087.602060790" watchObservedRunningTime="2026-06-09 12:12:58.320649094 +0000 UTC m=+1087.602466316" Jun 09 12:12:58.379914 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.379876 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-678947585c-hfkrn"] Jun 09 12:12:58.380114 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.380080 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-678947585c-hfkrn" podUID="0c68ffdf-2617-4f89-84ef-b663b658de3d" containerName="authorino" containerID="cri-o://883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f" gracePeriod=30 Jun 09 12:12:58.482868 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.482836 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:12:58.590014 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.589941 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:12:58.624453 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.624431 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:12:58.727667 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.727639 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfgvc\" (UniqueName: \"kubernetes.io/projected/0c68ffdf-2617-4f89-84ef-b663b658de3d-kube-api-access-rfgvc\") pod \"0c68ffdf-2617-4f89-84ef-b663b658de3d\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " Jun 09 12:12:58.727830 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.727697 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0c68ffdf-2617-4f89-84ef-b663b658de3d-oidc-ca\") pod \"0c68ffdf-2617-4f89-84ef-b663b658de3d\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " Jun 09 12:12:58.727830 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.727727 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c68ffdf-2617-4f89-84ef-b663b658de3d-tls-cert\") pod \"0c68ffdf-2617-4f89-84ef-b663b658de3d\" (UID: \"0c68ffdf-2617-4f89-84ef-b663b658de3d\") " Jun 09 12:12:58.729642 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.729613 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c68ffdf-2617-4f89-84ef-b663b658de3d-kube-api-access-rfgvc" (OuterVolumeSpecName: "kube-api-access-rfgvc") pod "0c68ffdf-2617-4f89-84ef-b663b658de3d" (UID: "0c68ffdf-2617-4f89-84ef-b663b658de3d"). InnerVolumeSpecName "kube-api-access-rfgvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:12:58.731832 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.731810 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c68ffdf-2617-4f89-84ef-b663b658de3d-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "0c68ffdf-2617-4f89-84ef-b663b658de3d" (UID: "0c68ffdf-2617-4f89-84ef-b663b658de3d"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 12:12:58.737394 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.737372 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c68ffdf-2617-4f89-84ef-b663b658de3d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0c68ffdf-2617-4f89-84ef-b663b658de3d" (UID: "0c68ffdf-2617-4f89-84ef-b663b658de3d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 12:12:58.828551 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.828524 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfgvc\" (UniqueName: \"kubernetes.io/projected/0c68ffdf-2617-4f89-84ef-b663b658de3d-kube-api-access-rfgvc\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:12:58.828551 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.828547 2550 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0c68ffdf-2617-4f89-84ef-b663b658de3d-oidc-ca\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:12:58.828698 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:58.828558 2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0c68ffdf-2617-4f89-84ef-b663b658de3d-tls-cert\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:12:59.305369 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.305329 2550 generic.go:356] "Generic (PLEG): container finished" podID="0c68ffdf-2617-4f89-84ef-b663b658de3d" containerID="883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f" exitCode=0 Jun 09 12:12:59.305369 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.305371 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678947585c-hfkrn" Jun 09 12:12:59.305908 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.305371 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678947585c-hfkrn" event={"ID":"0c68ffdf-2617-4f89-84ef-b663b658de3d","Type":"ContainerDied","Data":"883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f"} Jun 09 12:12:59.305908 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.305433 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678947585c-hfkrn" event={"ID":"0c68ffdf-2617-4f89-84ef-b663b658de3d","Type":"ContainerDied","Data":"786378e028d1e3b249132caea02f1a01a053bef5844b444613c8150cdc724003"} Jun 09 12:12:59.305908 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.305465 2550 scope.go:117] "RemoveContainer" containerID="883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f" Jun 09 12:12:59.314790 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.314773 2550 scope.go:117] "RemoveContainer" containerID="883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f" Jun 09 12:12:59.315048 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:12:59.315029 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f\": container with ID starting with 883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f not found: ID does not exist" containerID="883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f" Jun 09 12:12:59.315110 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.315056 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f"} err="failed to get container status \"883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f\": rpc error: code = NotFound desc = could not find container \"883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f\": container with ID starting with 883996600fb637cde5f5ba09a648fdbeb0d0a05ddc342a00dacb89a885e3a33f not found: ID does not exist" Jun 09 12:12:59.319858 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.319838 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-678947585c-hfkrn"] Jun 09 12:12:59.323452 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:12:59.323428 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-678947585c-hfkrn"] Jun 09 12:13:01.201497 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:01.201458 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c68ffdf-2617-4f89-84ef-b663b658de3d" path="/var/lib/kubelet/pods/0c68ffdf-2617-4f89-84ef-b663b658de3d/volumes" Jun 09 12:13:07.341909 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:07.341873 2550 generic.go:356] "Generic (PLEG): container finished" podID="8781f4c2-468c-4a2f-b74d-99d546493f13" containerID="81381aa85526f908ab00bdef9bf53359fc7045085678b0fb4c8d46f27684d86c" exitCode=0 Jun 09 12:13:07.342312 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:07.341945 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" event={"ID":"8781f4c2-468c-4a2f-b74d-99d546493f13","Type":"ContainerDied","Data":"81381aa85526f908ab00bdef9bf53359fc7045085678b0fb4c8d46f27684d86c"} Jun 09 12:13:08.348006 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:08.347958 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" event={"ID":"8781f4c2-468c-4a2f-b74d-99d546493f13","Type":"ContainerStarted","Data":"22b14b30cea724c45a5627f7e882eae3b528b4323838c33bc801899c9302d6b7"} Jun 09 12:13:08.369745 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:08.369683 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" podStartSLOduration=2.379709409 podStartE2EDuration="1m46.36966335s" totalImagesPullingTime="224.946405ms" totalInitContainerRuntime="1m43.765007536s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:13:08.367224961 +0000 UTC m=+1097.649042184" watchObservedRunningTime="2026-06-09 12:13:08.36966335 +0000 UTC m=+1097.651480574" Jun 09 12:13:12.501721 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:12.501686 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:13:12.502169 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:12.501769 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:13:12.514569 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:12.514541 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:13:13.378978 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:13.378946 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2" Jun 09 12:13:54.550095 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:54.550059 2550 generic.go:356] "Generic (PLEG): container finished" podID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" containerID="a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c" exitCode=1 Jun 09 12:13:54.550511 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:54.550135 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerDied","Data":"a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c"} Jun 09 12:13:54.550511 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:54.550192 2550 scope.go:117] "RemoveContainer" containerID="e778ba597992ad60dfbd8cb600ae302b8c59ffe2a05bf0aa918c79dc61e5b861" Jun 09 12:13:54.550616 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:54.550532 2550 scope.go:117] "RemoveContainer" containerID="a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c" Jun 09 12:13:54.550812 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:13:54.550793 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:13:56.035173 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:56.035134 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:13:56.035557 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:13:56.035500 2550 scope.go:117] "RemoveContainer" containerID="a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c" Jun 09 12:13:56.035667 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:13:56.035648 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:14:08.221092 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:08.221054 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:08.276932 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:08.276898 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:09.196601 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:09.196569 2550 scope.go:117] "RemoveContainer" containerID="a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c" Jun 09 12:14:09.617613 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:09.617528 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerStarted","Data":"3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230"} Jun 09 12:14:09.618085 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:09.617759 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:14:16.935226 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:16.935150 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:16.991039 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:16.991005 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:20.037952 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:20.037919 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:20.624392 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:20.624359 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:14:29.461919 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.461879 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:29.561427 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.561392 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869b98d549-2nv64"] Jun 09 12:14:29.561646 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.561622 2550 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-869b98d549-2nv64" podUID="b6a8d293-a6bf-4b62-a915-17a536bc3593" containerName="manager" containerID="cri-o://aaf2dc6d9fc5b838523417c310d0e555464e6cdec87556fde47ed12801edd834" gracePeriod=30 Jun 09 12:14:29.701974 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.701944 2550 generic.go:356] "Generic (PLEG): container finished" podID="b6a8d293-a6bf-4b62-a915-17a536bc3593" containerID="aaf2dc6d9fc5b838523417c310d0e555464e6cdec87556fde47ed12801edd834" exitCode=0 Jun 09 12:14:29.702125 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.702026 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869b98d549-2nv64" event={"ID":"b6a8d293-a6bf-4b62-a915-17a536bc3593","Type":"ContainerDied","Data":"aaf2dc6d9fc5b838523417c310d0e555464e6cdec87556fde47ed12801edd834"} Jun 09 12:14:29.809081 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.809058 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:14:29.871458 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.871428 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rknnf\" (UniqueName: \"kubernetes.io/projected/b6a8d293-a6bf-4b62-a915-17a536bc3593-kube-api-access-rknnf\") pod \"b6a8d293-a6bf-4b62-a915-17a536bc3593\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " Jun 09 12:14:29.871638 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.871500 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert\") pod \"b6a8d293-a6bf-4b62-a915-17a536bc3593\" (UID: \"b6a8d293-a6bf-4b62-a915-17a536bc3593\") " Jun 09 12:14:29.873519 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.873482 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a8d293-a6bf-4b62-a915-17a536bc3593-kube-api-access-rknnf" (OuterVolumeSpecName: "kube-api-access-rknnf") pod "b6a8d293-a6bf-4b62-a915-17a536bc3593" (UID: "b6a8d293-a6bf-4b62-a915-17a536bc3593"). InnerVolumeSpecName "kube-api-access-rknnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 12:14:29.873646 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.873628 2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b6a8d293-a6bf-4b62-a915-17a536bc3593" (UID: "b6a8d293-a6bf-4b62-a915-17a536bc3593"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 12:14:29.972330 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.972285 2550 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6a8d293-a6bf-4b62-a915-17a536bc3593-webhook-cert\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:14:29.972330 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:29.972322 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rknnf\" (UniqueName: \"kubernetes.io/projected/b6a8d293-a6bf-4b62-a915-17a536bc3593-kube-api-access-rknnf\") on node \"ip-10-0-139-166.ec2.internal\" DevicePath \"\"" Jun 09 12:14:30.707745 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:30.707711 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869b98d549-2nv64" event={"ID":"b6a8d293-a6bf-4b62-a915-17a536bc3593","Type":"ContainerDied","Data":"a9e7b0767ef957e60499d489db9d8c657298d576b494ca5dd1a1db1354896155"} Jun 09 12:14:30.708144 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:30.707765 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869b98d549-2nv64" Jun 09 12:14:30.708144 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:30.707796 2550 scope.go:117] "RemoveContainer" containerID="aaf2dc6d9fc5b838523417c310d0e555464e6cdec87556fde47ed12801edd834" Jun 09 12:14:30.728195 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:30.728173 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869b98d549-2nv64"] Jun 09 12:14:30.732184 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:30.732162 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-869b98d549-2nv64"] Jun 09 12:14:31.201557 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.201515 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6a8d293-a6bf-4b62-a915-17a536bc3593" path="/var/lib/kubelet/pods/b6a8d293-a6bf-4b62-a915-17a536bc3593/volumes" Jun 09 12:14:31.398032 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.397992 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:31.517283 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517212 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869b98d549-b57bp"] Jun 09 12:14:31.517497 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517485 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c68ffdf-2617-4f89-84ef-b663b658de3d" containerName="authorino" Jun 09 12:14:31.517537 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517498 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c68ffdf-2617-4f89-84ef-b663b658de3d" containerName="authorino" Jun 09 12:14:31.517537 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517512 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a8d293-a6bf-4b62-a915-17a536bc3593" containerName="manager" Jun 09 12:14:31.517537 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517517 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a8d293-a6bf-4b62-a915-17a536bc3593" containerName="manager" Jun 09 12:14:31.517631 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517565 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c68ffdf-2617-4f89-84ef-b663b658de3d" containerName="authorino" Jun 09 12:14:31.517631 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.517575 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a8d293-a6bf-4b62-a915-17a536bc3593" containerName="manager" Jun 09 12:14:31.521538 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.521520 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.525267 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.525242 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 12:14:31.525365 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.525248 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 12:14:31.525365 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.525286 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mz24z\"" Jun 09 12:14:31.529474 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.529450 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869b98d549-b57bp"] Jun 09 12:14:31.583800 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.583774 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcbsn\" (UniqueName: \"kubernetes.io/projected/e9deaece-d029-43bb-83bf-b5105cd0bb8c-kube-api-access-jcbsn\") pod \"maas-controller-869b98d549-b57bp\" (UID: \"e9deaece-d029-43bb-83bf-b5105cd0bb8c\") " pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.583919 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.583815 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9deaece-d029-43bb-83bf-b5105cd0bb8c-webhook-cert\") pod \"maas-controller-869b98d549-b57bp\" (UID: \"e9deaece-d029-43bb-83bf-b5105cd0bb8c\") " pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.684976 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.684947 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbsn\" (UniqueName: \"kubernetes.io/projected/e9deaece-d029-43bb-83bf-b5105cd0bb8c-kube-api-access-jcbsn\") pod \"maas-controller-869b98d549-b57bp\" (UID: \"e9deaece-d029-43bb-83bf-b5105cd0bb8c\") " pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.685083 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.684989 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9deaece-d029-43bb-83bf-b5105cd0bb8c-webhook-cert\") pod \"maas-controller-869b98d549-b57bp\" (UID: \"e9deaece-d029-43bb-83bf-b5105cd0bb8c\") " pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.687312 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.687284 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9deaece-d029-43bb-83bf-b5105cd0bb8c-webhook-cert\") pod \"maas-controller-869b98d549-b57bp\" (UID: \"e9deaece-d029-43bb-83bf-b5105cd0bb8c\") " pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.694404 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.694381 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbsn\" (UniqueName: \"kubernetes.io/projected/e9deaece-d029-43bb-83bf-b5105cd0bb8c-kube-api-access-jcbsn\") pod \"maas-controller-869b98d549-b57bp\" (UID: \"e9deaece-d029-43bb-83bf-b5105cd0bb8c\") " pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.832710 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.832643 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:31.959527 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:31.959489 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869b98d549-b57bp"] Jun 09 12:14:31.962804 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:14:31.962776 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9deaece_d029_43bb_83bf_b5105cd0bb8c.slice/crio-36449747a904d85387f07449d0d3e411c88b5893627ffe6edf76e896faa1ea2f WatchSource:0}: Error finding container 36449747a904d85387f07449d0d3e411c88b5893627ffe6edf76e896faa1ea2f: Status 404 returned error can't find the container with id 36449747a904d85387f07449d0d3e411c88b5893627ffe6edf76e896faa1ea2f Jun 09 12:14:32.718533 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:32.718453 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869b98d549-b57bp" event={"ID":"e9deaece-d029-43bb-83bf-b5105cd0bb8c","Type":"ContainerStarted","Data":"424eaa9425b2c9c3bace9f73770c6df1720f0746c01c265ee7e314c21bda1eeb"} Jun 09 12:14:32.718533 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:32.718527 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869b98d549-b57bp" event={"ID":"e9deaece-d029-43bb-83bf-b5105cd0bb8c","Type":"ContainerStarted","Data":"36449747a904d85387f07449d0d3e411c88b5893627ffe6edf76e896faa1ea2f"} Jun 09 12:14:32.718711 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:32.718618 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:32.736084 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:32.736042 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-869b98d549-b57bp" podStartSLOduration=1.322984982 podStartE2EDuration="1.736030255s" totalImagesPullingTime="413.045273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 12:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:14:32.734473814 +0000 UTC m=+1182.016291036" watchObservedRunningTime="2026-06-09 12:14:32.736030255 +0000 UTC m=+1182.017847478" Jun 09 12:14:43.728061 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:43.728031 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869b98d549-b57bp" Jun 09 12:14:53.471957 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:53.471927 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:14:58.054671 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:14:58.054639 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:11.337935 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:11.337898 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:11.441597 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:11.441565 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:19.856323 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:19.856289 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:25.766725 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:25.766688 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:25.825286 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:25.825254 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:57.858036 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:57.858004 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:15:58.954982 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:15:58.954948 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:10.119380 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:10.119348 2550 generic.go:356] "Generic (PLEG): container finished" podID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" exitCode=1 Jun 09 12:16:10.119852 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:10.119421 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerDied","Data":"3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230"} Jun 09 12:16:10.119852 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:10.119500 2550 scope.go:117] "RemoveContainer" containerID="a19155ad47d99cd6ec0cbe70b80e80c5642d78f4cf33d55951517b86ca93600c" Jun 09 12:16:10.119970 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:10.119853 2550 scope.go:117] "RemoveContainer" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" Jun 09 12:16:10.120092 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:16:10.120070 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:16:10.623431 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:10.623398 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:16:11.125527 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:11.125494 2550 scope.go:117] "RemoveContainer" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" Jun 09 12:16:11.126016 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:16:11.125700 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:16:11.873517 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:11.873487 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:16:12.129903 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:12.129880 2550 scope.go:117] "RemoveContainer" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" Jun 09 12:16:12.130260 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:16:12.130074 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:16:21.817768 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:21.817719 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:25.196890 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:25.196864 2550 scope.go:117] "RemoveContainer" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" Jun 09 12:16:25.197271 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:16:25.197086 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:16:27.879087 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:27.879049 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:27.985496 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:27.985462 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:28.079015 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:28.078977 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:28.237466 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:28.237394 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:28.333921 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:28.333866 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:39.196558 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:39.196528 2550 scope.go:117] "RemoveContainer" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" Jun 09 12:16:40.245805 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:40.245773 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerStarted","Data":"8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75"} Jun 09 12:16:40.246173 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:40.245972 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:16:44.753513 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:44.753474 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:44.868508 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:44.868478 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:44.973385 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:44.973355 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:50.861933 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:50.861900 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:50.920045 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:50.920012 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:51.251467 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:51.251436 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:16:58.999511 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:58.999476 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:59.096166 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:59.096131 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:59.196323 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:59.196276 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:16:59.297498 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:16:59.297425 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:15.706760 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:15.706657 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:15.806854 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:15.806817 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:15.914303 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:15.914272 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:21.777138 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:21.777097 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:21.883919 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:21.883874 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:42.131921 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:42.131885 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:42.232408 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:42.232377 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:42.343598 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:42.343561 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:48.057749 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:48.057708 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:48.175151 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:48.175120 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:48.276137 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:48.276107 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:48.406143 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:48.406110 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:48.480375 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:48.480343 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:54.322947 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:54.322911 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:17:54.393692 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:17:54.393664 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:18:08.786421 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:08.786385 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:18:39.742067 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:39.742031 2550 generic.go:356] "Generic (PLEG): container finished" podID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" exitCode=1 Jun 09 12:18:39.742582 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:39.742090 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerDied","Data":"8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75"} Jun 09 12:18:39.742582 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:39.742148 2550 scope.go:117] "RemoveContainer" containerID="3887111ebc26bf13fea20ba0894426d9779c394cdaf58307aaf3df8233185230" Jun 09 12:18:39.742582 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:39.742490 2550 scope.go:117] "RemoveContainer" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" Jun 09 12:18:39.742795 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:18:39.742766 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:18:41.249327 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:41.249249 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:18:41.249694 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:41.249522 2550 scope.go:117] "RemoveContainer" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" Jun 09 12:18:41.249694 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:18:41.249683 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:18:51.874223 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:51.874192 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:18:51.874650 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:18:51.874510 2550 scope.go:117] "RemoveContainer" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" Jun 09 12:18:51.874704 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:18:51.874686 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:19:05.196985 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:19:05.196950 2550 scope.go:117] "RemoveContainer" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" Jun 09 12:19:05.197396 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:19:05.197177 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:19:20.196137 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:19:20.196106 2550 scope.go:117] "RemoveContainer" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" Jun 09 12:19:20.921260 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:19:20.921223 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerStarted","Data":"ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf"} Jun 09 12:19:20.921457 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:19:20.921440 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:19:31.927892 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:19:31.927861 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:21:20.409123 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:20.409088 2550 generic.go:356] "Generic (PLEG): container finished" podID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" exitCode=1 Jun 09 12:21:20.409524 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:20.409158 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" event={"ID":"f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c","Type":"ContainerDied","Data":"ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf"} Jun 09 12:21:20.409524 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:20.409239 2550 scope.go:117] "RemoveContainer" containerID="8a42723eeb74127be4176b6da5f29bf89c07cad7abbfc4af7f39f75ab86a0e75" Jun 09 12:21:20.409524 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:20.409494 2550 scope.go:117] "RemoveContainer" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" Jun 09 12:21:20.409718 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:21:20.409701 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:21:21.926554 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:21.926518 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:21:21.926943 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:21.926893 2550 scope.go:117] "RemoveContainer" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" Jun 09 12:21:21.927103 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:21:21.927083 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:21:24.115370 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:24.115337 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:21:30.416399 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:30.416368 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5df6655c5b-7lcbv_db1e1072-5ade-445d-a904-cdf2214ecf69/authorino/0.log" Jun 09 12:21:31.873498 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:31.873455 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" Jun 09 12:21:31.873890 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:31.873848 2550 scope.go:117] "RemoveContainer" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" Jun 09 12:21:31.874013 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:21:31.873995 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:21:35.458693 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:35.458652 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-796d7c9c54-548kx_8b32e524-d3e9-4463-ae37-f14e94aa655e/maas-api/0.log" Jun 09 12:21:35.691623 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:35.691593 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869b98d549-b57bp_e9deaece-d029-43bb-83bf-b5105cd0bb8c/manager/0.log" Jun 09 12:21:36.293205 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:36.293180 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86976b546f-ljjtk_b0df865d-9ca9-489b-9a45-d7718ebb26fe/copy-manifests/0.log" Jun 09 12:21:36.304115 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:36.304089 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86976b546f-ljjtk_b0df865d-9ca9-489b-9a45-d7718ebb26fe/manager/0.log" Jun 09 12:21:37.702855 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:37.702829 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5df6655c5b-7lcbv_db1e1072-5ade-445d-a904-cdf2214ecf69/authorino/0.log" Jun 09 12:21:39.098369 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:39.098324 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd697648d-bvngf_06fe0e99-8910-4c42-a666-55e73b2b6347/kube-auth-proxy/0.log" Jun 09 12:21:39.462481 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:39.462446 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2r85z_6cd261fb-a5cd-4205-87d8-07100f9c227d/payload-pre-processing/0.log" Jun 09 12:21:39.579136 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:39.579104 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log" Jun 09 12:21:40.152920 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:40.152888 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-ff4rg_7335b52d-d0e1-4f7b-8901-99840e51fc82/storage-initializer/0.log" Jun 09 12:21:40.160003 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:40.159977 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-ff4rg_7335b52d-d0e1-4f7b-8901-99840e51fc82/main/0.log" Jun 09 12:21:40.392290 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:40.392260 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2_8781f4c2-468c-4a2f-b74d-99d546493f13/storage-initializer/0.log" Jun 09 12:21:40.404703 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:40.404637 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c7dfp2_8781f4c2-468c-4a2f-b74d-99d546493f13/main/0.log" Jun 09 12:21:42.196757 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:42.196669 2550 scope.go:117] "RemoveContainer" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" Jun 09 12:21:42.197123 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:21:42.196843 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:21:44.465448 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:44.465417 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:21:44.562042 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:44.562012 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" limit=1024 Jun 09 12:21:47.832413 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:47.832383 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-knwgt_d2098e65-cf15-4be3-b98c-4afe77fee72c/global-pull-secret-syncer/0.log" Jun 09 12:21:47.933008 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:47.932975 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5d8h9_7e3219bd-6dce-4959-ace5-290c62ed8a76/konnectivity-agent/0.log" Jun 09 12:21:48.056804 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:48.056773 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-166.ec2.internal_8d847be753db4224a797cf6deaab03dc/haproxy/0.log" Jun 09 12:21:53.057963 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:53.057926 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5df6655c5b-7lcbv_db1e1072-5ade-445d-a904-cdf2214ecf69/authorino/0.log" Jun 09 12:21:53.196492 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:53.196458 2550 scope.go:117] "RemoveContainer" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" Jun 09 12:21:53.196703 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:21:53.196680 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:21:55.698920 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:55.698893 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46r7b_59c27d58-f420-4474-9cd7-fb9c12b2046b/node-exporter/0.log" Jun 09 12:21:55.742146 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:55.742126 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46r7b_59c27d58-f420-4474-9cd7-fb9c12b2046b/kube-rbac-proxy/0.log" Jun 09 12:21:55.794751 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:55.794710 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46r7b_59c27d58-f420-4474-9cd7-fb9c12b2046b/init-textfile/0.log" Jun 09 12:21:58.556506 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.556473 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc"] Jun 09 12:21:58.559775 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.559728 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.563472 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.563447 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bn8ls\"/\"default-dockercfg-pl9tb\"" Jun 09 12:21:58.563593 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.563500 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bn8ls\"/\"openshift-service-ca.crt\"" Jun 09 12:21:58.564334 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.564315 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bn8ls\"/\"kube-root-ca.crt\"" Jun 09 12:21:58.569504 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.569481 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc"] Jun 09 12:21:58.707722 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.707686 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-podres\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.707722 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.707721 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-lib-modules\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.707977 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.707834 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n57z5\" (UniqueName: \"kubernetes.io/projected/531d3576-bfff-480d-81fd-e2bb0c1339c6-kube-api-access-n57z5\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.707977 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.707887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-sys\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.707977 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.707904 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-proc\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808473 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808403 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-lib-modules\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808473 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808454 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n57z5\" (UniqueName: \"kubernetes.io/projected/531d3576-bfff-480d-81fd-e2bb0c1339c6-kube-api-access-n57z5\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808684 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808491 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-sys\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808684 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808504 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-proc\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808684 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808530 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-podres\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808684 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808616 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-sys\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808684 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808617 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-lib-modules\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808684 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808619 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-proc\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.808936 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.808781 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/531d3576-bfff-480d-81fd-e2bb0c1339c6-podres\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.817106 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.817086 2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n57z5\" (UniqueName: \"kubernetes.io/projected/531d3576-bfff-480d-81fd-e2bb0c1339c6-kube-api-access-n57z5\") pod \"perf-node-gather-daemonset-tcrpc\" (UID: \"531d3576-bfff-480d-81fd-e2bb0c1339c6\") " pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:58.871168 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:58.871147 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:59.002803 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:59.002763 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc"] Jun 09 12:21:59.005563 ip-10-0-139-166 kubenswrapper[2550]: W0609 12:21:59.005535 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod531d3576_bfff_480d_81fd_e2bb0c1339c6.slice/crio-a20de968a48c2e70c4e46c1f4d7d7c93a19394ee48ade259c2b340448be4067f WatchSource:0}: Error finding container a20de968a48c2e70c4e46c1f4d7d7c93a19394ee48ade259c2b340448be4067f: Status 404 returned error can't find the container with id a20de968a48c2e70c4e46c1f4d7d7c93a19394ee48ade259c2b340448be4067f Jun 09 12:21:59.570190 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:59.570155 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" event={"ID":"531d3576-bfff-480d-81fd-e2bb0c1339c6","Type":"ContainerStarted","Data":"321982b21acf66c224a5d69393efd3aa15182de7c5512b93e348fb10bff0ca73"} Jun 09 12:21:59.570563 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:59.570209 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" event={"ID":"531d3576-bfff-480d-81fd-e2bb0c1339c6","Type":"ContainerStarted","Data":"a20de968a48c2e70c4e46c1f4d7d7c93a19394ee48ade259c2b340448be4067f"} Jun 09 12:21:59.570563 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:59.570272 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:21:59.585528 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:21:59.585476 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" podStartSLOduration=1.585464389 podStartE2EDuration="1.585464389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 12:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 12:21:59.585199741 +0000 UTC m=+1628.867016958" watchObservedRunningTime="2026-06-09 12:21:59.585464389 +0000 UTC m=+1628.867281612" Jun 09 12:22:01.450716 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:01.450671 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rhksv_7534b1a3-71fe-46df-8218-c8703fdf5f7b/dns/0.log" Jun 09 12:22:01.478518 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:01.478488 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rhksv_7534b1a3-71fe-46df-8218-c8703fdf5f7b/kube-rbac-proxy/0.log" Jun 09 12:22:01.506692 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:01.506663 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p9fst_f55d0bdb-53c4-4e01-b6e3-be9388a0de06/dns-node-resolver/0.log" Jun 09 12:22:02.199666 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:02.199638 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kzjmq_0d0a2e7e-9116-4e14-a83e-2708fd94c97b/node-ca/0.log" Jun 09 12:22:03.275230 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:03.275199 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd697648d-bvngf_06fe0e99-8910-4c42-a666-55e73b2b6347/kube-auth-proxy/0.log" Jun 09 12:22:03.406895 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:03.406867 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2r85z_6cd261fb-a5cd-4205-87d8-07100f9c227d/payload-pre-processing/0.log" Jun 09 12:22:03.434074 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:03.434048 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log" Jun 09 12:22:03.434220 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:03.434085 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log" Jun 09 12:22:04.120173 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:04.120143 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8wjnj_bc83f433-9d76-424b-9236-42bd78569b7c/serve-healthcheck-canary/0.log" Jun 09 12:22:04.196649 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:04.196616 2550 scope.go:117] "RemoveContainer" containerID="ca2724b8d0e930d4abde0217c8143341e50ee3d1344c4f02b087d5ad0e3156cf" Jun 09 12:22:04.196875 ip-10-0-139-166 kubenswrapper[2550]: E0609 12:22:04.196819 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-7mmmh_openshift-ingress(f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c)\"" pod="openshift-ingress/payload-processing-7c6c859784-7mmmh" podUID="f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c" Jun 09 12:22:04.847897 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:04.847859 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5k4g_56be4bd5-c956-4324-8b8e-d661773b4c84/kube-rbac-proxy/0.log" Jun 09 12:22:04.868934 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:04.868912 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5k4g_56be4bd5-c956-4324-8b8e-d661773b4c84/exporter/0.log" Jun 09 12:22:04.894174 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:04.894150 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z5k4g_56be4bd5-c956-4324-8b8e-d661773b4c84/extractor/0.log" Jun 09 12:22:05.583660 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:05.583636 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bn8ls/perf-node-gather-daemonset-tcrpc" Jun 09 12:22:07.622804 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:07.622776 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-796d7c9c54-548kx_8b32e524-d3e9-4463-ae37-f14e94aa655e/maas-api/0.log" Jun 09 12:22:07.691625 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:07.691597 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-869b98d549-b57bp_e9deaece-d029-43bb-83bf-b5105cd0bb8c/manager/0.log" Jun 09 12:22:08.016339 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:08.016315 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86976b546f-ljjtk_b0df865d-9ca9-489b-9a45-d7718ebb26fe/manager/0.log" Jun 09 12:22:08.046655 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:08.046631 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86976b546f-ljjtk_b0df865d-9ca9-489b-9a45-d7718ebb26fe/copy-manifests/0.log" Jun 09 12:22:09.751967 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:09.751935 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57f7d46b46-rn8gv_ae71faa1-1a03-4d63-95f6-f047e24b79b1/manager/0.log" Jun 09 12:22:12.768210 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:12.768176 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd697648d-bvngf_06fe0e99-8910-4c42-a666-55e73b2b6347/kube-auth-proxy/0.log" Jun 09 12:22:12.861565 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:12.861532 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2r85z_6cd261fb-a5cd-4205-87d8-07100f9c227d/payload-pre-processing/0.log" Jun 09 12:22:12.879562 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:12.879539 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log" Jun 09 12:22:12.880544 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:12.880526 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log" Jun 09 12:22:14.091112 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:14.091080 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fd697648d-bvngf_06fe0e99-8910-4c42-a666-55e73b2b6347/kube-auth-proxy/0.log" Jun 09 12:22:14.191262 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:14.191234 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2r85z_6cd261fb-a5cd-4205-87d8-07100f9c227d/payload-pre-processing/0.log" Jun 09 12:22:14.206199 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:14.206173 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log" Jun 09 12:22:14.206410 ip-10-0-139-166 kubenswrapper[2550]: I0609 12:22:14.206336 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-7mmmh_f3c66d2a-e7e3-4ec3-9ba2-dffa7429803c/payload-processing/4.log"