Jun 02 04:41:57.460471 ip-10-0-135-20 systemd[1]: Starting Kubernetes Kubelet... Jun 02 04:41:57.897300 ip-10-0-135-20 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 04:41:57.897300 ip-10-0-135-20 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 04:41:57.897300 ip-10-0-135-20 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 04:41:57.897300 ip-10-0-135-20 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 04:41:57.897300 ip-10-0-135-20 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 04:41:57.899299 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.899205 2566 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 04:41:57.906974 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906945 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 04:41:57.906974 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906970 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 04:41:57.906974 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906976 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 04:41:57.906974 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906979 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 04:41:57.906974 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906983 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906986 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906989 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906992 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906995 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.906998 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907000 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907003 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907006 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907008 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907011 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907013 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907016 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907019 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907021 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907023 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907026 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907029 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907032 2566 feature_gate.go:328] unrecognized feature gate: Example Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907034 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 04:41:57.907221 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907040 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907043 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907046 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907049 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907052 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907054 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907057 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907059 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907062 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907065 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907067 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907070 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907072 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907075 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907079 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907082 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907085 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907088 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907091 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907094 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 04:41:57.907686 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907097 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907099 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907102 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907104 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907107 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907109 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907112 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907114 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907117 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907120 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907123 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907126 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907128 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907131 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907133 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907147 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907151 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907153 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907156 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 04:41:57.908214 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907160 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907164 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907167 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907170 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907173 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907176 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907178 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907182 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907187 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907189 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907192 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907195 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907198 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907201 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907203 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907207 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907209 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907212 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907214 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907217 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 04:41:57.908677 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907219 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907222 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907224 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907636 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907641 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907644 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907647 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907650 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907653 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907655 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907658 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907660 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907663 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907666 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907668 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907671 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907674 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907677 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907679 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 04:41:57.909182 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907683 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907686 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907689 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907692 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907695 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907698 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907700 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907703 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907706 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907709 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907712 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907714 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907717 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907719 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907722 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907725 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907729 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907731 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907734 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 04:41:57.909639 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907736 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907739 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907741 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907744 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907746 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907749 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907752 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907755 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907757 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907760 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907763 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907765 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907768 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907771 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907773 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907776 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907779 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907782 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907785 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907788 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 04:41:57.910132 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907790 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907794 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907799 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907802 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907804 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907807 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907810 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907813 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907816 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907818 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907821 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907824 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907827 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907831 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907833 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907835 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907838 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907840 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907844 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 04:41:57.910636 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907847 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907850 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907852 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907855 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907857 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907860 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907862 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907865 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907868 2566 feature_gate.go:328] unrecognized feature gate: Example Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907870 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907873 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.907876 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907953 2566 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907961 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907967 2566 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907971 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907976 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907980 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907984 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907989 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907992 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 04:41:57.911107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907995 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.907998 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908001 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908005 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908020 2566 flags.go:64] FLAG: --cgroup-root="" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908024 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908027 2566 flags.go:64] FLAG: --client-ca-file="" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908030 2566 flags.go:64] FLAG: --cloud-config="" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908033 2566 flags.go:64] FLAG: --cloud-provider="external" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908036 2566 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908041 2566 flags.go:64] FLAG: --cluster-domain="" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908044 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908047 2566 flags.go:64] FLAG: --config-dir="" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908050 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908053 2566 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908057 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908061 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908064 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908068 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908071 2566 flags.go:64] FLAG: --contention-profiling="false" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908074 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908076 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908079 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908082 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908087 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 04:41:57.911649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908090 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908092 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908095 2566 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908099 2566 flags.go:64] FLAG: --enable-server="true" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908102 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908106 2566 flags.go:64] FLAG: --event-burst="100" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908111 2566 flags.go:64] FLAG: --event-qps="50" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908114 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908117 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908120 2566 flags.go:64] FLAG: --eviction-hard="" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908124 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908127 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908130 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908133 2566 flags.go:64] FLAG: --eviction-soft="" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908149 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908152 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908155 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908158 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908161 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908164 2566 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908167 2566 flags.go:64] FLAG: --feature-gates="" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908171 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908174 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908177 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908184 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908187 2566 flags.go:64] FLAG: --healthz-port="10248" Jun 02 04:41:57.912338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908190 2566 flags.go:64] FLAG: --help="false" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908193 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-135-20.ec2.internal" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908196 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908199 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908202 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908206 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908209 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908212 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908215 2566 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908218 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908221 2566 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908225 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908228 2566 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908231 2566 flags.go:64] FLAG: --kube-reserved="" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908234 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908237 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908240 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908243 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908246 2566 flags.go:64] FLAG: --lock-file="" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908249 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908252 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908254 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908260 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 04:41:57.912962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908263 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908266 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908269 2566 flags.go:64] FLAG: --logging-format="text" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908271 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908275 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908278 2566 flags.go:64] FLAG: --manifest-url="" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908281 2566 flags.go:64] FLAG: --manifest-url-header="" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908287 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908291 2566 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908295 2566 flags.go:64] FLAG: --max-pods="110" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908298 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908301 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908304 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908307 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908309 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908312 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908315 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908323 2566 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908326 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908329 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908333 2566 flags.go:64] FLAG: --pod-cidr="" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908336 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908341 2566 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908344 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 04:41:57.913536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908349 2566 flags.go:64] FLAG: --pods-per-core="0" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908351 2566 flags.go:64] FLAG: --port="10250" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908354 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908357 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00c2650fba51aadb4" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908361 2566 flags.go:64] FLAG: --qos-reserved="" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908364 2566 flags.go:64] FLAG: --read-only-port="10255" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908367 2566 flags.go:64] FLAG: --register-node="true" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908369 2566 flags.go:64] FLAG: --register-schedulable="true" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908372 2566 flags.go:64] FLAG: --register-with-taints="" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908376 2566 flags.go:64] FLAG: --registry-burst="10" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908379 2566 flags.go:64] FLAG: --registry-qps="5" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908381 2566 flags.go:64] FLAG: --reserved-cpus="" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908384 2566 flags.go:64] FLAG: --reserved-memory="" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908388 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908391 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908394 2566 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908398 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908401 2566 flags.go:64] FLAG: --runonce="false" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908404 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908407 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908410 2566 flags.go:64] FLAG: --seccomp-default="false" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908413 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908416 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908419 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908422 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908426 2566 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 04:41:57.914231 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908428 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908431 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908434 2566 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908437 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908441 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908444 2566 flags.go:64] FLAG: --system-cgroups="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908448 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908453 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908456 2566 flags.go:64] FLAG: --tls-cert-file="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908459 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908463 2566 flags.go:64] FLAG: --tls-min-version="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908466 2566 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908469 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908472 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908475 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908478 2566 flags.go:64] FLAG: --v="2" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908482 2566 flags.go:64] FLAG: --version="false" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908486 2566 flags.go:64] FLAG: --vmodule="" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908490 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908494 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908583 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908587 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908591 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908593 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 04:41:57.914855 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908596 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908599 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908601 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908604 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908606 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908611 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908614 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908617 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908620 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908623 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908626 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908629 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908632 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908635 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908639 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908642 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908644 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908647 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908650 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 04:41:57.915494 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908652 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908655 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908658 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908661 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908664 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908666 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908669 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908671 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908674 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908677 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908679 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908683 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908686 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908689 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908691 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908694 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908697 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908699 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908702 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 04:41:57.915991 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908705 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908707 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908710 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908712 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908715 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908717 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908720 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908723 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908726 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908729 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908732 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908734 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908737 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908740 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908742 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908745 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908747 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908750 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908752 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908755 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 04:41:57.916577 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908757 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908760 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908762 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908765 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908769 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908773 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908776 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908779 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908781 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908784 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908786 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908789 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908791 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908794 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908796 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908799 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908801 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908804 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908807 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908809 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 04:41:57.917076 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908812 2566 feature_gate.go:328] unrecognized feature gate: Example Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908815 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908818 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.908820 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.908826 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.915267 2566 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.915286 2566 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915336 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915342 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915345 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915349 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915351 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915355 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915357 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915360 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915363 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 04:41:57.917582 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915366 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915368 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915371 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915373 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915376 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915378 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915381 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915384 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915386 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915389 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915392 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915395 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915398 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915400 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915403 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915405 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915408 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915411 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915414 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915416 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 04:41:57.917982 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915419 2566 feature_gate.go:328] unrecognized feature gate: Example Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915423 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915427 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915429 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915432 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915434 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915437 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915440 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915442 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915445 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915448 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915450 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915453 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915455 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915458 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915460 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915463 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915465 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915468 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915472 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 04:41:57.918502 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915476 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915479 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915481 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915484 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915487 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915489 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915493 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915496 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915498 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915503 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915507 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915510 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915513 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915516 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915519 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915522 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915525 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915528 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915530 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 04:41:57.919000 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915533 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915536 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915538 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915541 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915543 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915546 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915548 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915551 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915553 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915556 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915558 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915561 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915563 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915566 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915568 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915571 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915573 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 04:41:57.919512 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915576 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.915581 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915681 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915685 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915688 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915691 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915695 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915699 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915703 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915706 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915709 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915712 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915716 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915719 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915722 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915724 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 04:41:57.919967 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915727 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915730 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915732 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915735 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915738 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915740 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915743 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915745 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915748 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915750 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915753 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915755 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915758 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915761 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915763 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915766 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915768 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915771 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915773 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915776 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 04:41:57.920406 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915778 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915781 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915784 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915786 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915789 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915791 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915794 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915798 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915800 2566 feature_gate.go:328] unrecognized feature gate: Example Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915804 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915807 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915810 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915812 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915815 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915818 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915820 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915822 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915825 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915827 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915830 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 04:41:57.920893 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915832 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915835 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915837 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915840 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915843 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915845 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915848 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915851 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915854 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915856 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915859 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915861 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915863 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915866 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915869 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915871 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915874 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915876 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915879 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 04:41:57.921401 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915881 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915884 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915887 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915890 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915892 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915895 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915897 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915900 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915902 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915905 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915907 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915910 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:57.915912 2566 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.915917 2566 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.916816 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 04:41:57.921861 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.918897 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 04:41:57.922297 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.919850 2566 server.go:1019] "Starting client certificate rotation" Jun 02 04:41:57.922297 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.919950 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 04:41:57.922297 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.919991 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 04:41:57.946126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.946102 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 04:41:57.949277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.949175 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 04:41:57.965519 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.965497 2566 log.go:25] "Validated CRI v1 runtime API" Jun 02 04:41:57.971966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.971945 2566 log.go:25] "Validated CRI v1 image API" Jun 02 04:41:57.973419 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.973391 2566 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 04:41:57.981072 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.981042 2566 fs.go:135] Filesystem UUIDs: map[7734bb7d-3377-4e74-a42e-7d02e6d5db1e:/dev/nvme0n1p3 78839409-4412-4c97-8972-a4937bc69696:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 02 04:41:57.981198 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.981069 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 04:41:57.981198 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.981114 2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 04:41:57.985655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.985526 2566 manager.go:217] Machine: {Timestamp:2026-06-02 04:41:57.984996031 +0000 UTC m=+0.408766285 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3084391 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d37e2a135d090aacd2d3fb70d2ea2 SystemUUID:ec2d37e2-a135-d090-aacd-2d3fb70d2ea2 BootID:a5d06108-a215-48aa-b9bc-44164a6429f0 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2a:00:35:16:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2a:00:35:16:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:90:f6:2d:5d:d9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 04:41:57.985655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.985641 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 04:41:57.985839 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.985747 2566 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 04:41:57.987759 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.987736 2566 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 04:41:57.987906 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.987762 2566 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 04:41:57.987952 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.987915 2566 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 04:41:57.987952 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.987924 2566 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 04:41:57.987952 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.987936 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 04:41:57.988048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.987953 2566 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 04:41:57.989365 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.989354 2566 state_mem.go:36] "Initialized new in-memory state store" Jun 02 04:41:57.989486 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.989477 2566 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 04:41:57.991606 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.991594 2566 kubelet.go:491] "Attempting to sync node with API server" Jun 02 04:41:57.991643 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.991611 2566 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 04:41:57.992282 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.992270 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 04:41:57.992322 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.992292 2566 kubelet.go:397] "Adding apiserver pod source" Jun 02 04:41:57.992322 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.992306 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 04:41:57.993793 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.993775 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 04:41:57.993861 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.993802 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 04:41:57.998526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:57.998375 2566 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 04:41:58.000470 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.000447 2566 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 04:41:58.002360 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002346 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002365 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002372 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002377 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002386 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002392 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002398 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002404 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002411 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002417 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002426 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 04:41:58.002431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.002431 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 04:41:58.003353 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.003342 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 04:41:58.003353 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.003352 2566 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 04:41:58.003975 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.003948 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 04:41:58.003975 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.003954 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 04:41:58.005897 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.005876 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rh94b" Jun 02 04:41:58.006663 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.006645 2566 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 04:41:58.006736 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.006690 2566 server.go:1295] "Started kubelet" Jun 02 04:41:58.006781 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.006744 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 04:41:58.007330 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.007263 2566 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 04:41:58.007414 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.007372 2566 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 04:41:58.007573 ip-10-0-135-20 systemd[1]: Started Kubernetes Kubelet. Jun 02 04:41:58.007650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.007570 2566 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 04:41:58.008513 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.008419 2566 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 04:41:58.008904 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.008890 2566 server.go:317] "Adding debug handlers to kubelet server" Jun 02 04:41:58.013672 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.013652 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 04:41:58.014267 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.014251 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 04:41:58.014562 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.014541 2566 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 04:41:58.014562 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.014563 2566 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 04:41:58.014732 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.014686 2566 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 04:41:58.014775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.014753 2566 reconstruct.go:97] "Volume reconstruction finished" Jun 02 04:41:58.014775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.014761 2566 reconciler.go:26] "Reconciler: start to sync state" Jun 02 04:41:58.015582 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.013436 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-20.ec2.internal.18b529eb3ca8faf6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-20.ec2.internal,UID:ip-10-0-135-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-20.ec2.internal,},FirstTimestamp:2026-06-02 04:41:58.00665983 +0000 UTC m=+0.430430089,LastTimestamp:2026-06-02 04:41:58.00665983 +0000 UTC m=+0.430430089,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-20.ec2.internal,}" Jun 02 04:41:58.015783 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.015514 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.015783 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.015777 2566 factory.go:55] Registering systemd factory Jun 02 04:41:58.015887 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.015791 2566 factory.go:223] Registration of the systemd container factory successfully Jun 02 04:41:58.015962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.015937 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rh94b" Jun 02 04:41:58.016246 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.016118 2566 factory.go:153] Registering CRI-O factory Jun 02 04:41:58.016246 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.016133 2566 factory.go:223] Registration of the crio container factory successfully Jun 02 04:41:58.016246 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.016198 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 04:41:58.016246 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.016218 2566 factory.go:103] Registering Raw factory Jun 02 04:41:58.016246 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.016233 2566 manager.go:1196] Started watching for new ooms in manager Jun 02 04:41:58.016774 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.016759 2566 manager.go:319] Starting recovery of all containers Jun 02 04:41:58.017725 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.017693 2566 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 04:41:58.021461 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.021429 2566 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 04:41:58.021568 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.021525 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 04:41:58.027424 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.027259 2566 manager.go:324] Recovery completed Jun 02 04:41:58.032474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.032457 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 04:41:58.035201 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.035187 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory" Jun 02 04:41:58.035258 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.035215 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 04:41:58.035258 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.035225 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID" Jun 02 04:41:58.036628 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.036614 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 04:41:58.036628 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.036627 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 04:41:58.036747 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.036647 2566 state_mem.go:36] "Initialized new in-memory state store" Jun 02 04:41:58.037571 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.037506 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-20.ec2.internal.18b529eb3e5c8021 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-20.ec2.internal,UID:ip-10-0-135-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-20.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-20.ec2.internal,},FirstTimestamp:2026-06-02 04:41:58.035202081 +0000 UTC m=+0.458972335,LastTimestamp:2026-06-02 04:41:58.035202081 +0000 UTC m=+0.458972335,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-20.ec2.internal,}" Jun 02 04:41:58.038783 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.038769 2566 policy_none.go:49] "None policy: Start" Jun 02 04:41:58.038856 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.038787 2566 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 04:41:58.038856 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.038801 2566 state_mem.go:35] "Initializing new in-memory state store" Jun 02 04:41:58.074764 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.074740 2566 manager.go:341] "Starting Device Plugin manager" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.074791 2566 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.074807 2566 server.go:85] "Starting device plugin registration server" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.075088 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.075101 2566 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.075192 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.075291 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.075304 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.075779 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 04:41:58.082631 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.075808 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.112990 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.112952 2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 04:41:58.114179 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.114161 2566 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 04:41:58.114179 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.114183 2566 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 04:41:58.114340 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.114206 2566 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 04:41:58.114340 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.114215 2566 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 04:41:58.114340 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.114261 2566 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 04:41:58.116936 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.116911 2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 04:41:58.175975 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.175893 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 04:41:58.176886 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.176867 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory" Jun 02 04:41:58.177011 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.176901 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 04:41:58.177011 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.176916 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID" Jun 02 04:41:58.177011 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.176945 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.185561 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.185544 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.185631 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.185569 2566 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-20.ec2.internal\": node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.202848 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.202819 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.215295 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.215268 2566 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"] Jun 02 04:41:58.215377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.215345 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 04:41:58.217124 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.217107 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory" Jun 02 04:41:58.217216 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.217151 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 04:41:58.217216 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.217162 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID" Jun 02 04:41:58.218323 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.218310 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 04:41:58.218472 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.218458 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.218508 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.218489 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 04:41:58.219087 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.219073 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory" Jun 02 04:41:58.219087 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.219080 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory" Jun 02 04:41:58.219206 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.219100 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 04:41:58.219206 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.219114 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID" Jun 02 04:41:58.219272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.219102 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 04:41:58.219272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.219233 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID" Jun 02 04:41:58.220432 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.220418 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.220499 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.220441 2566 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 04:41:58.221917 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.221901 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory" Jun 02 04:41:58.221917 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.221928 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 04:41:58.222050 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.221938 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID" Jun 02 04:41:58.237709 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.237691 2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-20.ec2.internal\" not found" node="ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.241631 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.241613 2566 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-20.ec2.internal\" not found" node="ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.303819 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.303785 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.316124 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.316096 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc5453b3f2ec30ae0a361186cb47e71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"8fc5453b3f2ec30ae0a361186cb47e71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.316226 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.316128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12dc1b6bf110d069bf03b7ee9f029d82-config\") pod \"kube-apiserver-proxy-ip-10-0-135-20.ec2.internal\" (UID: \"12dc1b6bf110d069bf03b7ee9f029d82\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.316259 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.316235 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fc5453b3f2ec30ae0a361186cb47e71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"8fc5453b3f2ec30ae0a361186cb47e71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.404071 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.404017 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.417395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.417365 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fc5453b3f2ec30ae0a361186cb47e71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"8fc5453b3f2ec30ae0a361186cb47e71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.417483 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.417403 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc5453b3f2ec30ae0a361186cb47e71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"8fc5453b3f2ec30ae0a361186cb47e71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.417483 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.417421 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12dc1b6bf110d069bf03b7ee9f029d82-config\") pod \"kube-apiserver-proxy-ip-10-0-135-20.ec2.internal\" (UID: \"12dc1b6bf110d069bf03b7ee9f029d82\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.417584 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.417489 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fc5453b3f2ec30ae0a361186cb47e71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"8fc5453b3f2ec30ae0a361186cb47e71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.417584 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.417532 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/12dc1b6bf110d069bf03b7ee9f029d82-config\") pod \"kube-apiserver-proxy-ip-10-0-135-20.ec2.internal\" (UID: \"12dc1b6bf110d069bf03b7ee9f029d82\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.417584 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.417554 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc5453b3f2ec30ae0a361186cb47e71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"8fc5453b3f2ec30ae0a361186cb47e71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.504869 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.504793 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.540283 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.540255 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.544923 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.544902 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" Jun 02 04:41:58.605484 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.605447 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.705984 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.705950 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.806513 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.806430 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.906963 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:58.906930 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:58.919219 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.919195 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 04:41:58.919370 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:58.919347 2566 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 04:41:59.002191 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:59.002154 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dc1b6bf110d069bf03b7ee9f029d82.slice/crio-e1e4557e2267e5858bcae5d10d3c18abb60bafa61511b1a8534f12291345f0f3 WatchSource:0}: Error finding container e1e4557e2267e5858bcae5d10d3c18abb60bafa61511b1a8534f12291345f0f3: Status 404 returned error can't find the container with id e1e4557e2267e5858bcae5d10d3c18abb60bafa61511b1a8534f12291345f0f3 Jun 02 04:41:59.002454 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:41:59.002424 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc5453b3f2ec30ae0a361186cb47e71.slice/crio-5ca51da4660530bd25d01c77f7ad1eec30f39a1957e5609b59e2daa2a6f0ed70 WatchSource:0}: Error finding container 5ca51da4660530bd25d01c77f7ad1eec30f39a1957e5609b59e2daa2a6f0ed70: Status 404 returned error can't find the container with id 5ca51da4660530bd25d01c77f7ad1eec30f39a1957e5609b59e2daa2a6f0ed70 Jun 02 04:41:59.006976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.006952 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 04:41:59.007061 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:59.007045 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:59.014011 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.013992 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 04:41:59.023943 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.023916 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 04:36:58 +0000 UTC" deadline="2027-10-26 04:54:19.809205297 +0000 UTC" Jun 02 04:41:59.023943 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.023941 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12264h12m20.785267341s" Jun 02 04:41:59.030550 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.030531 2566 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 04:41:59.060996 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.060929 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8mxw5" Jun 02 04:41:59.072223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.072201 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8mxw5" Jun 02 04:41:59.107726 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:41:59.107695 2566 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found" Jun 02 04:41:59.117737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.117690 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" event={"ID":"12dc1b6bf110d069bf03b7ee9f029d82","Type":"ContainerStarted","Data":"e1e4557e2267e5858bcae5d10d3c18abb60bafa61511b1a8534f12291345f0f3"} Jun 02 04:41:59.118523 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.118502 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" event={"ID":"8fc5453b3f2ec30ae0a361186cb47e71","Type":"ContainerStarted","Data":"5ca51da4660530bd25d01c77f7ad1eec30f39a1957e5609b59e2daa2a6f0ed70"} Jun 02 04:41:59.159895 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.159869 2566 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 04:41:59.215016 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.214985 2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" Jun 02 04:41:59.227602 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.227579 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 04:41:59.229250 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.229233 2566 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" Jun 02 04:41:59.238265 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.238249 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 04:41:59.427911 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.427831 2566 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 04:41:59.604431 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.604397 2566 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 04:41:59.993576 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.993531 2566 apiserver.go:52] "Watching apiserver" Jun 02 04:41:59.998986 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:41:59.998958 2566 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 04:42:00.000208 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.000177 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-6gtwr","openshift-network-diagnostics/network-check-target-qpm4h","openshift-network-operator/iptables-alerter-8gw6c","kube-system/konnectivity-agent-zpqvp","kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv","openshift-dns/node-resolver-z4v4x","openshift-image-registry/node-ca-ndsfj","openshift-multus/multus-4gjpt","openshift-multus/network-metrics-daemon-l8f45","openshift-ovn-kubernetes/ovnkube-node-hjcp9","openshift-cluster-node-tuning-operator/tuned-mbd7s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"] Jun 02 04:42:00.002256 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.002235 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.005113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.004900 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.005113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.004941 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:00.005113 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.005021 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:00.005113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.005092 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m7t4t\"" Jun 02 04:42:00.005387 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.005099 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.006192 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.006131 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.006408 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.006393 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.007461 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.007441 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.008451 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008432 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 04:42:00.008544 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008489 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.008544 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008495 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hz5kw\"" Jun 02 04:42:00.008650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008435 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.008650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008637 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 04:42:00.008784 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008769 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hgsft\"" Jun 02 04:42:00.008878 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.008860 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 04:42:00.009266 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.009250 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.009612 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.009591 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kzzcd\"" Jun 02 04:42:00.010049 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.009910 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.010049 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.009938 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.010049 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.009944 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 04:42:00.010597 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.010566 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.011781 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.011763 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.011937 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.011926 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 04:42:00.012099 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012085 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.012184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012166 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 04:42:00.012247 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012237 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fgl8l\"" Jun 02 04:42:00.012444 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012416 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 04:42:00.012552 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012537 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.012552 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012545 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 04:42:00.012764 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012748 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kv2dd\"" Jun 02 04:42:00.012832 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012750 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.012832 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.012787 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.014480 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.013922 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.014480 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.014022 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:00.017180 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.017062 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.018124 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.018072 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6lj2s\"" Jun 02 04:42:00.018282 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.018265 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 04:42:00.018769 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.018738 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019566 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019589 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019569 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019591 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019682 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019694 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qv7cw\"" Jun 02 04:42:00.019798 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.019718 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 04:42:00.021865 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.021850 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 04:42:00.021988 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.021855 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 04:42:00.023248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.023232 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w92zj\"" Jun 02 04:42:00.026556 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026533 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.026663 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/34e12dd2-625a-412d-b26c-206d4f40b16e-serviceca\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.026663 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krxsg\" (UniqueName: \"kubernetes.io/projected/34e12dd2-625a-412d-b26c-206d4f40b16e-kube-api-access-krxsg\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.026663 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-daemon-config\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.026663 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-slash\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-cni-multus\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovn-node-metrics-cert\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-cni-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-cni-bin\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026831 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-env-overrides\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.026864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxzk8\" (UniqueName: \"kubernetes.io/projected/d85df90f-0704-466b-bfc3-be341c63a82a-kube-api-access-hxzk8\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-etc-selinux\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-netns\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.026994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d85df90f-0704-466b-bfc3-be341c63a82a-hosts-file\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-sys-fs\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m8jh\" (UniqueName: \"kubernetes.io/projected/395e222e-8be8-4bc2-aaeb-92ab5148229d-kube-api-access-2m8jh\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-os-release\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027117 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kpvc\" (UniqueName: \"kubernetes.io/projected/8d48b463-ab69-454c-b60c-9251bd561dd7-kube-api-access-9kpvc\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-kubelet\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-systemd\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027215 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-run-ovn-kubernetes\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-cnibin\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-run-netns\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovnkube-config\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.027538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027553 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-conf-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-multus-certs\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027603 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d48b463-ab69-454c-b60c-9251bd561dd7-cni-binary-copy\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lrl7\" (UniqueName: \"kubernetes.io/projected/0a740bc8-62c1-4218-b1b0-36e4e59d3089-kube-api-access-9lrl7\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8776f2db-ee73-4f3d-a9e3-4a339c6e51a9-agent-certs\") pod \"konnectivity-agent-zpqvp\" (UID: \"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9\") " pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-registration-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.027934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.027728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-device-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.028279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028087 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgm8\" (UniqueName: \"kubernetes.io/projected/480a31aa-30b5-46d6-bf37-602349111aca-kube-api-access-9jgm8\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.028279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028165 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-system-cni-dir\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.028279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-cni-bin\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.028279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-log-socket\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.028279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a740bc8-62c1-4218-b1b0-36e4e59d3089-host-slash\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.028279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028278 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8776f2db-ee73-4f3d-a9e3-4a339c6e51a9-konnectivity-ca\") pod \"konnectivity-agent-zpqvp\" (UID: \"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9\") " pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-system-cni-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028330 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-systemd-units\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-os-release\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-hostroot\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028426 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-etc-kubernetes\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-etc-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.028514 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-node-log\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028557 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snp7r\" (UniqueName: \"kubernetes.io/projected/6d79be2a-3c4d-4eef-8793-27c3d4718282-kube-api-access-snp7r\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a740bc8-62c1-4218-b1b0-36e4e59d3089-iptables-alerter-script\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-socket-dir-parent\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-kubelet\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-cnibin\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/34e12dd2-625a-412d-b26c-206d4f40b16e-host\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d85df90f-0704-466b-bfc3-be341c63a82a-tmp-dir\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028751 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-socket-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.028815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-cni-netd\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-var-lib-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028907 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-ovn\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028934 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-k8s-cni-cncf-io\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.028983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovnkube-script-lib\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.029158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.029009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b79f\" (UniqueName: \"kubernetes.io/projected/b05f924c-4cf3-4db7-bb80-baa2e515793e-kube-api-access-9b79f\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.073357 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.073321 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 04:36:59 +0000 UTC" deadline="2028-01-18 02:10:55.851703419 +0000 UTC" Jun 02 04:42:00.073357 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.073351 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14277h28m55.778355137s" Jun 02 04:42:00.116475 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.116435 2566 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 04:42:00.119630 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.119604 2566 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 04:42:00.129411 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129378 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-log-socket\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129416 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a740bc8-62c1-4218-b1b0-36e4e59d3089-host-slash\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129444 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8776f2db-ee73-4f3d-a9e3-4a339c6e51a9-konnectivity-ca\") pod \"konnectivity-agent-zpqvp\" (UID: \"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9\") " pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129472 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-lib-modules\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129499 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-system-cni-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129519 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a740bc8-62c1-4218-b1b0-36e4e59d3089-host-slash\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129526 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-systemd-units\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129552 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-os-release\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.129582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129576 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-hostroot\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129594 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-systemd-units\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129603 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-etc-kubernetes\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129550 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-log-socket\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129633 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-etc-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129597 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-system-cni-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129643 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-hostroot\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129651 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-etc-kubernetes\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129686 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129712 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-node-log\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129755 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129791 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-etc-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129760 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-node-log\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129812 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-snp7r\" (UniqueName: \"kubernetes.io/projected/6d79be2a-3c4d-4eef-8793-27c3d4718282-kube-api-access-snp7r\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129828 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-os-release\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129835 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a740bc8-62c1-4218-b1b0-36e4e59d3089-iptables-alerter-script\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129866 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-socket-dir-parent\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.129902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129893 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-kubelet\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129917 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-cnibin\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129939 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/34e12dd2-625a-412d-b26c-206d4f40b16e-host\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129963 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d85df90f-0704-466b-bfc3-be341c63a82a-tmp-dir\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129980 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8776f2db-ee73-4f3d-a9e3-4a339c6e51a9-konnectivity-ca\") pod \"konnectivity-agent-zpqvp\" (UID: \"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9\") " pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129990 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-socket-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.129991 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-socket-dir-parent\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130011 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/34e12dd2-625a-412d-b26c-206d4f40b16e-host\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysconfig\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130041 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-cnibin\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130055 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-kubelet\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130068 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-systemd\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130102 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130127 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-socket-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130127 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-cni-netd\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-run\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130181 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-cni-netd\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.130742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-sys\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130227 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-var-lib-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130260 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-ovn\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130286 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130295 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-var-lib-openvswitch\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130333 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-ovn\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-kubernetes\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130306 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d85df90f-0704-466b-bfc3-be341c63a82a-tmp-dir\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130364 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130396 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysctl-conf\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-host\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130466 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-k8s-cni-cncf-io\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130492 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovnkube-script-lib\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130518 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9b79f\" (UniqueName: \"kubernetes.io/projected/b05f924c-4cf3-4db7-bb80-baa2e515793e-kube-api-access-9b79f\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-k8s-cni-cncf-io\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130544 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130578 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/34e12dd2-625a-412d-b26c-206d4f40b16e-serviceca\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.131535 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130603 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krxsg\" (UniqueName: \"kubernetes.io/projected/34e12dd2-625a-412d-b26c-206d4f40b16e-kube-api-access-krxsg\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130628 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-daemon-config\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130651 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-slash\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130676 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130701 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-cni-multus\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130725 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovn-node-metrics-cert\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-var-lib-kubelet\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130775 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-cni-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130803 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-cni-bin\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130826 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-env-overrides\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130883 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hxzk8\" (UniqueName: \"kubernetes.io/projected/d85df90f-0704-466b-bfc3-be341c63a82a-kube-api-access-hxzk8\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130910 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-etc-selinux\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130932 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130958 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.130986 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-netns\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131041 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d85df90f-0704-466b-bfc3-be341c63a82a-hosts-file\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/34e12dd2-625a-412d-b26c-206d4f40b16e-serviceca\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.132420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131068 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-sys-fs\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131062 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a740bc8-62c1-4218-b1b0-36e4e59d3089-iptables-alerter-script\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131090 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovnkube-script-lib\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klgs6\" (UniqueName: \"kubernetes.io/projected/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-kube-api-access-klgs6\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131154 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2m8jh\" (UniqueName: \"kubernetes.io/projected/395e222e-8be8-4bc2-aaeb-92ab5148229d-kube-api-access-2m8jh\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131207 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-tuned\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131231 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131235 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-os-release\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131267 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131285 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9kpvc\" (UniqueName: \"kubernetes.io/projected/8d48b463-ab69-454c-b60c-9251bd561dd7-kube-api-access-9kpvc\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131302 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-os-release\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131314 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-kubelet\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131327 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-slash\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131341 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-systemd\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131367 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-run-ovn-kubernetes\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131388 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-sys-fs\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131396 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-cnibin\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131429 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-cni-multus\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131435 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-run-netns\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131447 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131475 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d85df90f-0704-466b-bfc3-be341c63a82a-hosts-file\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-netns\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131493 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovnkube-config\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131523 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-kubelet\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysctl-d\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131537 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-cnibin\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131537 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-env-overrides\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131550 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-tmp\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131400 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-etc-selinux\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131561 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-run-systemd\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131584 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131583 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/395e222e-8be8-4bc2-aaeb-92ab5148229d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131626 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-cni-bin\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131689 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-run-netns\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131699 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-run-ovn-kubernetes\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.133800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131733 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-conf-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131764 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-cni-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131807 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-multus-certs\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131813 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.131820 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131822 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-conf-dir\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131856 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-run-multus-certs\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.131951 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:00.631911607 +0000 UTC m=+3.055681865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.131976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-modprobe-d\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132024 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8d48b463-ab69-454c-b60c-9251bd561dd7-multus-daemon-config\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132049 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d48b463-ab69-454c-b60c-9251bd561dd7-cni-binary-copy\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132082 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132097 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovnkube-config\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132110 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9lrl7\" (UniqueName: \"kubernetes.io/projected/0a740bc8-62c1-4218-b1b0-36e4e59d3089-kube-api-access-9lrl7\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132166 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8776f2db-ee73-4f3d-a9e3-4a339c6e51a9-agent-certs\") pod \"konnectivity-agent-zpqvp\" (UID: \"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9\") " pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132195 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-registration-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132234 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d79be2a-3c4d-4eef-8793-27c3d4718282-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.134455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132223 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-device-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132291 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-device-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132302 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgm8\" (UniqueName: \"kubernetes.io/projected/480a31aa-30b5-46d6-bf37-602349111aca-kube-api-access-9jgm8\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132347 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/480a31aa-30b5-46d6-bf37-602349111aca-registration-dir\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132377 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-system-cni-dir\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132406 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-cni-bin\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132476 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d48b463-ab69-454c-b60c-9251bd561dd7-host-var-lib-cni-bin\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132492 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/395e222e-8be8-4bc2-aaeb-92ab5148229d-system-cni-dir\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.135223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.132548 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d48b463-ab69-454c-b60c-9251bd561dd7-cni-binary-copy\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.135660 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.135623 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d79be2a-3c4d-4eef-8793-27c3d4718282-ovn-node-metrics-cert\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.135834 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.135749 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8776f2db-ee73-4f3d-a9e3-4a339c6e51a9-agent-certs\") pod \"konnectivity-agent-zpqvp\" (UID: \"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9\") " pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.139628 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.139603 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:00.139628 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.139632 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:00.139808 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.139647 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:00.139808 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.139733 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:00.639712904 +0000 UTC m=+3.063483147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:00.145977 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.145948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b79f\" (UniqueName: \"kubernetes.io/projected/b05f924c-4cf3-4db7-bb80-baa2e515793e-kube-api-access-9b79f\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.147050 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.147028 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snp7r\" (UniqueName: \"kubernetes.io/projected/6d79be2a-3c4d-4eef-8793-27c3d4718282-kube-api-access-snp7r\") pod \"ovnkube-node-hjcp9\" (UID: \"6d79be2a-3c4d-4eef-8793-27c3d4718282\") " pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.158435 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.158404 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxzk8\" (UniqueName: \"kubernetes.io/projected/d85df90f-0704-466b-bfc3-be341c63a82a-kube-api-access-hxzk8\") pod \"node-resolver-z4v4x\" (UID: \"d85df90f-0704-466b-bfc3-be341c63a82a\") " pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.158662 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.158424 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lrl7\" (UniqueName: \"kubernetes.io/projected/0a740bc8-62c1-4218-b1b0-36e4e59d3089-kube-api-access-9lrl7\") pod \"iptables-alerter-8gw6c\" (UID: \"0a740bc8-62c1-4218-b1b0-36e4e59d3089\") " pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.158778 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.158431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgm8\" (UniqueName: \"kubernetes.io/projected/480a31aa-30b5-46d6-bf37-602349111aca-kube-api-access-9jgm8\") pod \"aws-ebs-csi-driver-node-857rv\" (UID: \"480a31aa-30b5-46d6-bf37-602349111aca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.159272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.159238 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kpvc\" (UniqueName: \"kubernetes.io/projected/8d48b463-ab69-454c-b60c-9251bd561dd7-kube-api-access-9kpvc\") pod \"multus-4gjpt\" (UID: \"8d48b463-ab69-454c-b60c-9251bd561dd7\") " pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.159443 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.159420 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krxsg\" (UniqueName: \"kubernetes.io/projected/34e12dd2-625a-412d-b26c-206d4f40b16e-kube-api-access-krxsg\") pod \"node-ca-ndsfj\" (UID: \"34e12dd2-625a-412d-b26c-206d4f40b16e\") " pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.159603 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.159584 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m8jh\" (UniqueName: \"kubernetes.io/projected/395e222e-8be8-4bc2-aaeb-92ab5148229d-kube-api-access-2m8jh\") pod \"multus-additional-cni-plugins-6gtwr\" (UID: \"395e222e-8be8-4bc2-aaeb-92ab5148229d\") " pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.233376 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233343 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysctl-d\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233376 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233377 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-tmp\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233401 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-modprobe-d\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233435 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-lib-modules\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233471 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysconfig\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233494 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-systemd\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233534 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-run\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233544 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysctl-d\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233571 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysconfig\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233588 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-systemd\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.233594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233555 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-sys\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233599 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-run\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233610 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-sys\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-lib-modules\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233549 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-modprobe-d\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233636 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-kubernetes\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233693 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysctl-conf\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233718 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-host\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233724 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-kubernetes\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233754 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-var-lib-kubelet\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233787 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-host\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233797 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-klgs6\" (UniqueName: \"kubernetes.io/projected/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-kube-api-access-klgs6\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233824 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-tuned\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233838 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-var-lib-kubelet\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.234126 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.233850 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-sysctl-conf\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.235797 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.235772 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-tmp\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.235938 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.235918 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-etc-tuned\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.242462 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.242441 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klgs6\" (UniqueName: \"kubernetes.io/projected/e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c-kube-api-access-klgs6\") pod \"tuned-mbd7s\" (UID: \"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c\") " pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.313414 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.313318 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4v4x" Jun 02 04:42:00.326230 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.326205 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8gw6c" Jun 02 04:42:00.331884 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.331863 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:00.335550 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.335530 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" Jun 02 04:42:00.341084 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.341061 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" Jun 02 04:42:00.346639 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.346621 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ndsfj" Jun 02 04:42:00.352171 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.352148 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4gjpt" Jun 02 04:42:00.358770 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.358746 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:00.363417 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.363396 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" Jun 02 04:42:00.545096 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.545070 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480a31aa_30b5_46d6_bf37_602349111aca.slice/crio-635989f0a6ee4df1192f548efc9e81702de516cb1d2d1957a97c6af11c1ddb64 WatchSource:0}: Error finding container 635989f0a6ee4df1192f548efc9e81702de516cb1d2d1957a97c6af11c1ddb64: Status 404 returned error can't find the container with id 635989f0a6ee4df1192f548efc9e81702de516cb1d2d1957a97c6af11c1ddb64 Jun 02 04:42:00.545791 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.545762 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fe5a8e_bed8_4e14_86fd_f0eeca089d7c.slice/crio-2ae529bfbd5db2bc03986415ee5ead11486d0325b6556b9e1b2aafdd1bf22bfc WatchSource:0}: Error finding container 2ae529bfbd5db2bc03986415ee5ead11486d0325b6556b9e1b2aafdd1bf22bfc: Status 404 returned error can't find the container with id 2ae529bfbd5db2bc03986415ee5ead11486d0325b6556b9e1b2aafdd1bf22bfc Jun 02 04:42:00.546961 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.546898 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d79be2a_3c4d_4eef_8793_27c3d4718282.slice/crio-5a97d9cdf97c37138b996744833b3c51938c33b8f0c8b29edc3fba2e8dc01ce3 WatchSource:0}: Error finding container 5a97d9cdf97c37138b996744833b3c51938c33b8f0c8b29edc3fba2e8dc01ce3: Status 404 returned error can't find the container with id 5a97d9cdf97c37138b996744833b3c51938c33b8f0c8b29edc3fba2e8dc01ce3 Jun 02 04:42:00.547902 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.547854 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395e222e_8be8_4bc2_aaeb_92ab5148229d.slice/crio-926a94e0cae0bdf30103bea4970bb2e40d21203456f49da42ff2deca1f877d9c WatchSource:0}: Error finding container 926a94e0cae0bdf30103bea4970bb2e40d21203456f49da42ff2deca1f877d9c: Status 404 returned error can't find the container with id 926a94e0cae0bdf30103bea4970bb2e40d21203456f49da42ff2deca1f877d9c Jun 02 04:42:00.551579 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.551484 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85df90f_0704_466b_bfc3_be341c63a82a.slice/crio-d9eca384e9142ecbd34eebc5037819a7f1d1143e5cdd6d96f5b20469098b387c WatchSource:0}: Error finding container d9eca384e9142ecbd34eebc5037819a7f1d1143e5cdd6d96f5b20469098b387c: Status 404 returned error can't find the container with id d9eca384e9142ecbd34eebc5037819a7f1d1143e5cdd6d96f5b20469098b387c Jun 02 04:42:00.552378 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.552354 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d48b463_ab69_454c_b60c_9251bd561dd7.slice/crio-af037c807a049bdf85d868959b17895f1c321a9b9024bea3375e284992137018 WatchSource:0}: Error finding container af037c807a049bdf85d868959b17895f1c321a9b9024bea3375e284992137018: Status 404 returned error can't find the container with id af037c807a049bdf85d868959b17895f1c321a9b9024bea3375e284992137018 Jun 02 04:42:00.553296 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.553279 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a740bc8_62c1_4218_b1b0_36e4e59d3089.slice/crio-4766b87efbb040b18100f6f7167e1ab3b00a3941b4275415e94681f3ad2e951c WatchSource:0}: Error finding container 4766b87efbb040b18100f6f7167e1ab3b00a3941b4275415e94681f3ad2e951c: Status 404 returned error can't find the container with id 4766b87efbb040b18100f6f7167e1ab3b00a3941b4275415e94681f3ad2e951c Jun 02 04:42:00.554296 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:00.554271 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8776f2db_ee73_4f3d_a9e3_4a339c6e51a9.slice/crio-e687d295bb4ca22190565c0a86e622f08a6996f16ee493ec66afb7efa88b6da7 WatchSource:0}: Error finding container e687d295bb4ca22190565c0a86e622f08a6996f16ee493ec66afb7efa88b6da7: Status 404 returned error can't find the container with id e687d295bb4ca22190565c0a86e622f08a6996f16ee493ec66afb7efa88b6da7 Jun 02 04:42:00.637039 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.636868 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:00.637198 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.637022 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:00.637198 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.637191 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:01.637170204 +0000 UTC m=+4.060940458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:00.737801 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:00.737770 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:00.737985 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.737902 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:00.737985 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.737923 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:00.737985 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.737935 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:00.738121 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:00.737991 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:01.737972402 +0000 UTC m=+4.161742644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:01.074395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.074300 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 04:36:59 +0000 UTC" deadline="2028-03-01 03:19:32.352842246 +0000 UTC" Jun 02 04:42:01.074395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.074342 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15310h37m31.278503647s" Jun 02 04:42:01.132093 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.132051 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerStarted","Data":"926a94e0cae0bdf30103bea4970bb2e40d21203456f49da42ff2deca1f877d9c"} Jun 02 04:42:01.138010 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.137970 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"5a97d9cdf97c37138b996744833b3c51938c33b8f0c8b29edc3fba2e8dc01ce3"} Jun 02 04:42:01.158199 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.158161 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" event={"ID":"480a31aa-30b5-46d6-bf37-602349111aca","Type":"ContainerStarted","Data":"635989f0a6ee4df1192f548efc9e81702de516cb1d2d1957a97c6af11c1ddb64"} Jun 02 04:42:01.178739 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.178684 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" event={"ID":"12dc1b6bf110d069bf03b7ee9f029d82","Type":"ContainerStarted","Data":"6c7de5822f1e4eeb1fc8eb8af7e19c8cfb62553ac231d22e058ee36ac9a5ec88"} Jun 02 04:42:01.197446 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.196648 2566 generic.go:358] "Generic (PLEG): container finished" podID="8fc5453b3f2ec30ae0a361186cb47e71" containerID="310bff8bd3bb8eb65a1a97b66b8c8f5a0e4914325aa2b5ca5dd4abaf2a7be1df" exitCode=0 Jun 02 04:42:01.197446 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.196751 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" event={"ID":"8fc5453b3f2ec30ae0a361186cb47e71","Type":"ContainerDied","Data":"310bff8bd3bb8eb65a1a97b66b8c8f5a0e4914325aa2b5ca5dd4abaf2a7be1df"} Jun 02 04:42:01.204775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.204577 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zpqvp" event={"ID":"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9","Type":"ContainerStarted","Data":"e687d295bb4ca22190565c0a86e622f08a6996f16ee493ec66afb7efa88b6da7"} Jun 02 04:42:01.204954 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.204912 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" podStartSLOduration=2.204897441 podStartE2EDuration="2.204897441s" podCreationTimestamp="2026-06-02 04:41:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:42:01.202371735 +0000 UTC m=+3.626142003" watchObservedRunningTime="2026-06-02 04:42:01.204897441 +0000 UTC m=+3.628667707" Jun 02 04:42:01.217606 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.217554 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8gw6c" event={"ID":"0a740bc8-62c1-4218-b1b0-36e4e59d3089","Type":"ContainerStarted","Data":"4766b87efbb040b18100f6f7167e1ab3b00a3941b4275415e94681f3ad2e951c"} Jun 02 04:42:01.230178 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.228910 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" event={"ID":"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c","Type":"ContainerStarted","Data":"2ae529bfbd5db2bc03986415ee5ead11486d0325b6556b9e1b2aafdd1bf22bfc"} Jun 02 04:42:01.233331 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.233276 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ndsfj" event={"ID":"34e12dd2-625a-412d-b26c-206d4f40b16e","Type":"ContainerStarted","Data":"ec5be201e12b464d6e99006fa6de19a4f775b7dd02bb0e0ed6dbc626f15a0d48"} Jun 02 04:42:01.247869 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.247835 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4gjpt" event={"ID":"8d48b463-ab69-454c-b60c-9251bd561dd7","Type":"ContainerStarted","Data":"af037c807a049bdf85d868959b17895f1c321a9b9024bea3375e284992137018"} Jun 02 04:42:01.257774 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.257732 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4v4x" event={"ID":"d85df90f-0704-466b-bfc3-be341c63a82a","Type":"ContainerStarted","Data":"d9eca384e9142ecbd34eebc5037819a7f1d1143e5cdd6d96f5b20469098b387c"} Jun 02 04:42:01.645555 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.645516 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:01.645771 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:01.645671 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:01.645771 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:01.645736 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:03.645717049 +0000 UTC m=+6.069487304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:01.746988 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:01.746937 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:01.747202 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:01.747125 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:01.747202 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:01.747160 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:01.747202 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:01.747174 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:01.747379 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:01.747234 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:03.747213864 +0000 UTC m=+6.170984112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:02.118578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:02.117623 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:02.118578 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:02.117760 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:02.118578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:02.118290 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:02.118578 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:02.118459 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:02.297033 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:02.296995 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" event={"ID":"8fc5453b3f2ec30ae0a361186cb47e71","Type":"ContainerStarted","Data":"3e6fb749dd0f21dc2de9d59585cbd7947d8dc81e1e1db08f652977865befe5d2"} Jun 02 04:42:02.315582 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:02.315515 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" podStartSLOduration=3.315494315 podStartE2EDuration="3.315494315s" podCreationTimestamp="2026-06-02 04:41:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:42:02.314743329 +0000 UTC m=+4.738513594" watchObservedRunningTime="2026-06-02 04:42:02.315494315 +0000 UTC m=+4.739264578" Jun 02 04:42:03.663035 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:03.662321 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:03.663035 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:03.662464 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:03.663035 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:03.662549 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:07.662528341 +0000 UTC m=+10.086298600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:03.763607 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:03.763387 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:03.763784 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:03.763695 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:03.763784 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:03.763716 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:03.763784 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:03.763730 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:03.763946 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:03.763796 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:07.763778149 +0000 UTC m=+10.187548398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:04.116210 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:04.116117 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:04.116362 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:04.116150 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:04.116362 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:04.116265 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:04.116362 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:04.116323 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:06.114698 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:06.114659 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:06.115095 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:06.114809 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:06.115259 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:06.115233 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:06.115373 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:06.115350 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:07.696878 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:07.696834 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:07.697488 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:07.697001 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:07.697488 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:07.697087 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:15.697065936 +0000 UTC m=+18.120836182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:07.797656 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:07.797618 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:07.797841 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:07.797785 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:07.797841 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:07.797802 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:07.797841 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:07.797811 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:07.797999 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:07.797867 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:15.797853899 +0000 UTC m=+18.221624140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:08.116496 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:08.116405 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:08.117063 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:08.117026 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:08.117654 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:08.117512 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:08.117654 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:08.117612 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:10.115263 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:10.115222 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:10.115263 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:10.115245 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:10.115812 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:10.115349 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:10.115812 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:10.115421 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:12.115513 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:12.115475 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:12.115513 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:12.115515 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:12.116047 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:12.115623 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:12.116047 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:12.115739 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:14.114562 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:14.114521 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:14.115031 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:14.114522 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:14.115031 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:14.114635 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:14.115031 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:14.114691 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:15.760927 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:15.760889 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:15.761361 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:15.761014 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:15.761361 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:15.761071 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:31.761056924 +0000 UTC m=+34.184827165 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:15.862115 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:15.862074 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:15.862306 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:15.862269 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:15.862306 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:15.862292 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:15.862306 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:15.862302 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:15.862430 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:15.862362 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:31.862343016 +0000 UTC m=+34.286113258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:16.115355 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:16.115318 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:16.115544 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:16.115319 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:16.115544 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:16.115443 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:16.115655 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:16.115560 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:18.115378 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:18.115335 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:18.115880 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:18.115437 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:18.115880 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:18.115467 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:18.115880 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:18.115520 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:19.329520 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.329166 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4v4x" event={"ID":"d85df90f-0704-466b-bfc3-be341c63a82a","Type":"ContainerStarted","Data":"0b167e4dbc38bde2b2ca65d157daf650a3684218d19b02d307f88815c13d2531"} Jun 02 04:42:19.330586 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.330554 2566 generic.go:358] "Generic (PLEG): container finished" podID="395e222e-8be8-4bc2-aaeb-92ab5148229d" containerID="619d181f8789937b285477c8415e500366e4a2a24533a3837c5285a78a631e39" exitCode=0 Jun 02 04:42:19.330696 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.330629 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerDied","Data":"619d181f8789937b285477c8415e500366e4a2a24533a3837c5285a78a631e39"} Jun 02 04:42:19.332196 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.332182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:42:19.332504 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.332485 2566 generic.go:358] "Generic (PLEG): container finished" podID="6d79be2a-3c4d-4eef-8793-27c3d4718282" containerID="c711feee41df47e4b28f778ce35b61a58b9f8103e4731f7e646907374eeccbce" exitCode=1 Jun 02 04:42:19.332583 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.332542 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"420c787e79dc2def3e953b66962812e10d2c66bc9724b0f6c11eb9f714527f32"} Jun 02 04:42:19.332583 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.332560 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerDied","Data":"c711feee41df47e4b28f778ce35b61a58b9f8103e4731f7e646907374eeccbce"} Jun 02 04:42:19.332583 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.332570 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"c76b6ae000fa3c9b2a5e3787656372cba0e66590817e3514cb1d3deb27d32337"} Jun 02 04:42:19.333994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.333953 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" event={"ID":"480a31aa-30b5-46d6-bf37-602349111aca","Type":"ContainerStarted","Data":"f65481aba2ca41b792ae5d09890ce5ef1133d58b648b07a943bf281db1f689d5"} Jun 02 04:42:19.335314 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.335279 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zpqvp" event={"ID":"8776f2db-ee73-4f3d-a9e3-4a339c6e51a9","Type":"ContainerStarted","Data":"93b74dd2e6d1de6e4d4c767113c550efa2ceff39775884a9ca4b6762617c6d01"} Jun 02 04:42:19.337931 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.337891 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" event={"ID":"e7fe5a8e-bed8-4e14-86fd-f0eeca089d7c","Type":"ContainerStarted","Data":"1ecdca8126feefe58ac99909966047b4cf1e6c5afafd51f8d577a4ff67a04392"} Jun 02 04:42:19.339646 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.339620 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ndsfj" event={"ID":"34e12dd2-625a-412d-b26c-206d4f40b16e","Type":"ContainerStarted","Data":"a541ad5bc013636cdec1532ebd02cf1ef68188d6419ed2eb52846f9ff90e2544"} Jun 02 04:42:19.340933 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.340914 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4gjpt" event={"ID":"8d48b463-ab69-454c-b60c-9251bd561dd7","Type":"ContainerStarted","Data":"d8f1f66016ff02d2a70860d416e30f161d6508c69111097fc0a13bee24901b03"} Jun 02 04:42:19.345127 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.345091 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-z4v4x" podStartSLOduration=3.211811314 podStartE2EDuration="21.345079846s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.553115956 +0000 UTC m=+2.976886201" lastFinishedPulling="2026-06-02 04:42:18.686384478 +0000 UTC m=+21.110154733" observedRunningTime="2026-06-02 04:42:19.345025973 +0000 UTC m=+21.768796236" watchObservedRunningTime="2026-06-02 04:42:19.345079846 +0000 UTC m=+21.768850109" Jun 02 04:42:19.376351 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.376296 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zpqvp" podStartSLOduration=3.250312649 podStartE2EDuration="21.376276608s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.560716169 +0000 UTC m=+2.984486423" lastFinishedPulling="2026-06-02 04:42:18.68668014 +0000 UTC m=+21.110450382" observedRunningTime="2026-06-02 04:42:19.358623702 +0000 UTC m=+21.782393965" watchObservedRunningTime="2026-06-02 04:42:19.376276608 +0000 UTC m=+21.800046861" Jun 02 04:42:19.376874 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.376843 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mbd7s" podStartSLOduration=3.23680755 podStartE2EDuration="21.376834876s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.547761065 +0000 UTC m=+2.971531305" lastFinishedPulling="2026-06-02 04:42:18.687788378 +0000 UTC m=+21.111558631" observedRunningTime="2026-06-02 04:42:19.376241619 +0000 UTC m=+21.800011882" watchObservedRunningTime="2026-06-02 04:42:19.376834876 +0000 UTC m=+21.800605138" Jun 02 04:42:19.431214 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.431151 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ndsfj" podStartSLOduration=11.414006407 podStartE2EDuration="21.431122924s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.562290041 +0000 UTC m=+2.986060282" lastFinishedPulling="2026-06-02 04:42:10.579406545 +0000 UTC m=+13.003176799" observedRunningTime="2026-06-02 04:42:19.41097768 +0000 UTC m=+21.834747946" watchObservedRunningTime="2026-06-02 04:42:19.431122924 +0000 UTC m=+21.854893187" Jun 02 04:42:19.431387 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.431245 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4gjpt" podStartSLOduration=3.289265714 podStartE2EDuration="21.431238528s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.556327428 +0000 UTC m=+2.980097686" lastFinishedPulling="2026-06-02 04:42:18.698300256 +0000 UTC m=+21.122070500" observedRunningTime="2026-06-02 04:42:19.430403654 +0000 UTC m=+21.854173942" watchObservedRunningTime="2026-06-02 04:42:19.431238528 +0000 UTC m=+21.855008790" Jun 02 04:42:19.900858 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:19.900828 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 04:42:20.087539 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.087422 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T04:42:19.900854265Z","UUID":"42348cf1-d8fc-4f3e-b383-652a436bfd39","Handler":null,"Name":"","Endpoint":""} Jun 02 04:42:20.089249 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.089228 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 04:42:20.089348 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.089255 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 04:42:20.115056 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.115025 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:20.115233 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:20.115129 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:20.115233 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.115193 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:20.115327 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:20.115304 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:20.344819 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.344727 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8gw6c" event={"ID":"0a740bc8-62c1-4218-b1b0-36e4e59d3089","Type":"ContainerStarted","Data":"0e4ed0c56814b188159b762719d9f8d0bc891403fcd2baca6c40a81d9749b550"} Jun 02 04:42:20.348010 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.347979 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:42:20.348473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.348406 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"693dd7ccf37b2bd2105a5bceb8d5ebe03b5114142232d086d1c0ebd20d8bb78f"} Jun 02 04:42:20.348473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.348441 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"de42183e6b42598bb930ed1ace04bc296da1ea315cecbfa5de41f3883ffdba3e"} Jun 02 04:42:20.348473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.348454 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"be3a51a85aba08902746cdf6fcdc4cd918a1f307810774f3ca97b1b9e20de307"} Jun 02 04:42:20.350349 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.350235 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" event={"ID":"480a31aa-30b5-46d6-bf37-602349111aca","Type":"ContainerStarted","Data":"388ae033eea011113f12d3b6ce3f041c0fce8951a04511435daf248f13185dd3"} Jun 02 04:42:20.361539 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:20.361493 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8gw6c" podStartSLOduration=4.25376132 podStartE2EDuration="22.361480293s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.555889919 +0000 UTC m=+2.979660161" lastFinishedPulling="2026-06-02 04:42:18.663608878 +0000 UTC m=+21.087379134" observedRunningTime="2026-06-02 04:42:20.361160875 +0000 UTC m=+22.784931136" watchObservedRunningTime="2026-06-02 04:42:20.361480293 +0000 UTC m=+22.785250555" Jun 02 04:42:21.354953 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:21.354862 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" event={"ID":"480a31aa-30b5-46d6-bf37-602349111aca","Type":"ContainerStarted","Data":"d22073d8124d5c0bf936477e507f775dd9fa14bbb24985d7aadc8c1e5fdc671c"} Jun 02 04:42:21.376198 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:21.376150 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-857rv" podStartSLOduration=2.820986542 podStartE2EDuration="23.376122094s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.54708405 +0000 UTC m=+2.970854308" lastFinishedPulling="2026-06-02 04:42:21.102219604 +0000 UTC m=+23.525989860" observedRunningTime="2026-06-02 04:42:21.375803977 +0000 UTC m=+23.799574253" watchObservedRunningTime="2026-06-02 04:42:21.376122094 +0000 UTC m=+23.799892362" Jun 02 04:42:22.115127 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:22.115040 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:22.115370 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:22.115040 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:22.115370 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:22.115183 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:22.115370 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:22.115268 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:22.360395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:22.360235 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:42:22.360827 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:22.360769 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"46a900b98949ad34167b2a952c3a24f919f4690ecc9ac5c7551e6ce3a4438a55"} Jun 02 04:42:24.114785 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.114645 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:24.116100 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:24.114786 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:24.116100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.114837 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:24.116100 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:24.114951 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:24.214346 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.214162 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:24.214723 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.214702 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:24.365469 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.365369 2566 generic.go:358] "Generic (PLEG): container finished" podID="395e222e-8be8-4bc2-aaeb-92ab5148229d" containerID="62672041253aecaf5999c7f653cc685da4d57777f7135413655ccab0b0513a1b" exitCode=0 Jun 02 04:42:24.365469 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.365442 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerDied","Data":"62672041253aecaf5999c7f653cc685da4d57777f7135413655ccab0b0513a1b"} Jun 02 04:42:24.368544 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.368525 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:42:24.368885 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.368865 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"ee304989ee9ee4aa62ac71f7583357385487d290befcbf7525e90ae4f9a26558"} Jun 02 04:42:24.369197 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.369149 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:24.369197 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.369185 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:24.369197 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.369200 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:24.369364 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.369212 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:24.369404 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.369365 2566 scope.go:117] "RemoveContainer" containerID="c711feee41df47e4b28f778ce35b61a58b9f8103e4731f7e646907374eeccbce" Jun 02 04:42:24.369695 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.369676 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zpqvp" Jun 02 04:42:24.386082 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.386060 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:24.386703 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:24.386680 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:25.372611 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.372582 2566 generic.go:358] "Generic (PLEG): container finished" podID="395e222e-8be8-4bc2-aaeb-92ab5148229d" containerID="8e2aa88400d56614d84b558e0f2b2159cdf0b909248e8dda17eb62b9fed64b7c" exitCode=0 Jun 02 04:42:25.373109 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.372669 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerDied","Data":"8e2aa88400d56614d84b558e0f2b2159cdf0b909248e8dda17eb62b9fed64b7c"} Jun 02 04:42:25.377849 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.377831 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:42:25.378201 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.378181 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" event={"ID":"6d79be2a-3c4d-4eef-8793-27c3d4718282","Type":"ContainerStarted","Data":"6bfa6f254dd84f2a8e2bf6c711b9f1334cf52ae13224829ccd6f8f44a612f4d4"} Jun 02 04:42:25.424572 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.424514 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" podStartSLOduration=9.03407809 podStartE2EDuration="27.42449505s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.549782728 +0000 UTC m=+2.973552986" lastFinishedPulling="2026-06-02 04:42:18.940199695 +0000 UTC m=+21.363969946" observedRunningTime="2026-06-02 04:42:25.424256379 +0000 UTC m=+27.848026642" watchObservedRunningTime="2026-06-02 04:42:25.42449505 +0000 UTC m=+27.848265316" Jun 02 04:42:25.969194 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.969163 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l8f45"] Jun 02 04:42:25.969360 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.969315 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:25.969443 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:25.969417 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:25.971839 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.971814 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qpm4h"] Jun 02 04:42:25.971970 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:25.971928 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:25.972034 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:25.972013 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:26.381834 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:26.381804 2566 generic.go:358] "Generic (PLEG): container finished" podID="395e222e-8be8-4bc2-aaeb-92ab5148229d" containerID="01ece796cd50cddff7058c8dc2d960259cd1bff1bfcee0057e71f6a3aee1c96d" exitCode=0 Jun 02 04:42:26.382242 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:26.381892 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerDied","Data":"01ece796cd50cddff7058c8dc2d960259cd1bff1bfcee0057e71f6a3aee1c96d"} Jun 02 04:42:27.115170 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:27.115117 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:27.115330 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:27.115284 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:28.116172 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:28.115665 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:28.116172 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:28.115774 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:29.114493 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:29.114454 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:29.114677 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:29.114607 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:30.115193 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:30.115164 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:30.115854 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:30.115289 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:31.114573 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:31.114541 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:31.114849 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.114663 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l8f45" podUID="b05f924c-4cf3-4db7-bb80-baa2e515793e" Jun 02 04:42:31.777976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:31.777929 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:31.778572 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.778097 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:31.778572 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.778194 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs podName:b05f924c-4cf3-4db7-bb80-baa2e515793e nodeName:}" failed. No retries permitted until 2026-06-02 04:43:03.778172135 +0000 UTC m=+66.201942376 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs") pod "network-metrics-daemon-l8f45" (UID: "b05f924c-4cf3-4db7-bb80-baa2e515793e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 04:42:31.879004 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:31.878967 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:31.879206 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.879110 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 04:42:31.879206 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.879127 2566 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 04:42:31.879206 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.879152 2566 projected.go:194] Error preparing data for projected volume kube-api-access-846t7 for pod openshift-network-diagnostics/network-check-target-qpm4h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:31.879347 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:31.879213 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7 podName:6a73057d-6f6b-4e14-97a7-a6e755a2cfa0 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:03.879193093 +0000 UTC m=+66.302963335 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-846t7" (UniqueName: "kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7") pod "network-check-target-qpm4h" (UID: "6a73057d-6f6b-4e14-97a7-a6e755a2cfa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 04:42:32.114707 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.114657 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:32.114897 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:32.114813 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qpm4h" podUID="6a73057d-6f6b-4e14-97a7-a6e755a2cfa0" Jun 02 04:42:32.453992 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.453892 2566 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeReady" Jun 02 04:42:32.454184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.454057 2566 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 04:42:32.500590 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.500558 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sjg59"] Jun 02 04:42:32.561525 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.561484 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-czq5w"] Jun 02 04:42:32.561721 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.561700 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.564649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.564623 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 04:42:32.564804 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.564663 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bzxgr\"" Jun 02 04:42:32.564804 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.564631 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 04:42:32.576747 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.576725 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sjg59"] Jun 02 04:42:32.576857 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.576757 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-czq5w"] Jun 02 04:42:32.576916 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.576898 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:32.579272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.579234 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 04:42:32.579400 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.579341 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 04:42:32.579557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.579535 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 04:42:32.579657 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.579587 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jhq2g\"" Jun 02 04:42:32.685292 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.685252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7mrv\" (UniqueName: \"kubernetes.io/projected/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-kube-api-access-t7mrv\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.685482 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.685314 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:32.685482 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.685362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-config-volume\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.685482 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.685398 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.685482 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.685418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw8qn\" (UniqueName: \"kubernetes.io/projected/16f61d36-7264-4685-811e-9bdfa53612e8-kube-api-access-jw8qn\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:32.685650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.685488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-tmp-dir\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.786658 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.786569 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:32.786658 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.786617 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-config-volume\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:32.786712 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.786739 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:32.786775 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert podName:16f61d36-7264-4685-811e-9bdfa53612e8 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:33.286758409 +0000 UTC m=+35.710528650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert") pod "ingress-canary-czq5w" (UID: "16f61d36-7264-4685-811e-9bdfa53612e8") : secret "canary-serving-cert" not found Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.786801 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jw8qn\" (UniqueName: \"kubernetes.io/projected/16f61d36-7264-4685-811e-9bdfa53612e8-kube-api-access-jw8qn\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:32.786824 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.786839 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-tmp-dir\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:32.786873 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls podName:8a5ad758-ecd6-46a7-88d2-d17354c1b7d5 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:33.286857138 +0000 UTC m=+35.710627392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls") pod "dns-default-sjg59" (UID: "8a5ad758-ecd6-46a7-88d2-d17354c1b7d5") : secret "dns-default-metrics-tls" not found Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.786918 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t7mrv\" (UniqueName: \"kubernetes.io/projected/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-kube-api-access-t7mrv\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.787203 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-tmp-dir\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.787275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.787246 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-config-volume\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.796829 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.796806 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7mrv\" (UniqueName: \"kubernetes.io/projected/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-kube-api-access-t7mrv\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:32.796954 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:32.796889 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw8qn\" (UniqueName: \"kubernetes.io/projected/16f61d36-7264-4685-811e-9bdfa53612e8-kube-api-access-jw8qn\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:33.115190 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.115008 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:42:33.117972 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.117950 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c2frh\"" Jun 02 04:42:33.118112 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.117957 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 04:42:33.291371 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.291337 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:33.291526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.291392 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:33.291526 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:33.291489 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 04:42:33.291526 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:33.291503 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 04:42:33.291640 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:33.291563 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls podName:8a5ad758-ecd6-46a7-88d2-d17354c1b7d5 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:34.291543315 +0000 UTC m=+36.715313565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls") pod "dns-default-sjg59" (UID: "8a5ad758-ecd6-46a7-88d2-d17354c1b7d5") : secret "dns-default-metrics-tls" not found Jun 02 04:42:33.291640 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:33.291579 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert podName:16f61d36-7264-4685-811e-9bdfa53612e8 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:34.291571737 +0000 UTC m=+36.715341977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert") pod "ingress-canary-czq5w" (UID: "16f61d36-7264-4685-811e-9bdfa53612e8") : secret "canary-serving-cert" not found Jun 02 04:42:33.398579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.398544 2566 generic.go:358] "Generic (PLEG): container finished" podID="395e222e-8be8-4bc2-aaeb-92ab5148229d" containerID="733e03b5f954d2360a3a4d20520e694f9913c884c3b9281d4ba691042a0b78ca" exitCode=0 Jun 02 04:42:33.398744 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:33.398599 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerDied","Data":"733e03b5f954d2360a3a4d20520e694f9913c884c3b9281d4ba691042a0b78ca"} Jun 02 04:42:34.114954 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.114917 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:42:34.117622 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.117595 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 04:42:34.117770 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.117635 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rvl8h\"" Jun 02 04:42:34.117770 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.117598 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 04:42:34.298497 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.298461 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:34.298497 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.298509 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:34.298739 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:34.298610 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 04:42:34.298739 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:34.298634 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 04:42:34.298739 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:34.298672 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert podName:16f61d36-7264-4685-811e-9bdfa53612e8 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:36.298655753 +0000 UTC m=+38.722426000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert") pod "ingress-canary-czq5w" (UID: "16f61d36-7264-4685-811e-9bdfa53612e8") : secret "canary-serving-cert" not found Jun 02 04:42:34.298739 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:34.298686 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls podName:8a5ad758-ecd6-46a7-88d2-d17354c1b7d5 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:36.29868082 +0000 UTC m=+38.722451060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls") pod "dns-default-sjg59" (UID: "8a5ad758-ecd6-46a7-88d2-d17354c1b7d5") : secret "dns-default-metrics-tls" not found Jun 02 04:42:34.402676 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.402589 2566 generic.go:358] "Generic (PLEG): container finished" podID="395e222e-8be8-4bc2-aaeb-92ab5148229d" containerID="b2336921e5a3c3fadbfee71f45cedcce3608dc49b6cd1a4fa827a2092615452f" exitCode=0 Jun 02 04:42:34.402676 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.402653 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerDied","Data":"b2336921e5a3c3fadbfee71f45cedcce3608dc49b6cd1a4fa827a2092615452f"} Jun 02 04:42:34.980885 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.980848 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl"] Jun 02 04:42:34.989091 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.989063 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" Jun 02 04:42:34.991571 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.991539 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 02 04:42:34.991867 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.991841 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 02 04:42:34.992034 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.991887 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-skblt\"" Jun 02 04:42:34.992508 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:34.992485 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl"] Jun 02 04:42:35.105050 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.105013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75k2l\" (UniqueName: \"kubernetes.io/projected/e4c2faa7-ddc4-407f-806f-071afd1ef8c7-kube-api-access-75k2l\") pod \"migrator-96c6f749-5rkhl\" (UID: \"e4c2faa7-ddc4-407f-806f-071afd1ef8c7\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" Jun 02 04:42:35.206223 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.206186 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75k2l\" (UniqueName: \"kubernetes.io/projected/e4c2faa7-ddc4-407f-806f-071afd1ef8c7-kube-api-access-75k2l\") pod \"migrator-96c6f749-5rkhl\" (UID: \"e4c2faa7-ddc4-407f-806f-071afd1ef8c7\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" Jun 02 04:42:35.216474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.216450 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75k2l\" (UniqueName: \"kubernetes.io/projected/e4c2faa7-ddc4-407f-806f-071afd1ef8c7-kube-api-access-75k2l\") pod \"migrator-96c6f749-5rkhl\" (UID: \"e4c2faa7-ddc4-407f-806f-071afd1ef8c7\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" Jun 02 04:42:35.298854 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.298767 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" Jun 02 04:42:35.408588 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.408554 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" event={"ID":"395e222e-8be8-4bc2-aaeb-92ab5148229d","Type":"ContainerStarted","Data":"42344b2a47287ceaf95069367a1be723942f7b336a6be6b8849bd4736ef2790f"} Jun 02 04:42:35.441024 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.440974 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6gtwr" podStartSLOduration=4.9748395720000005 podStartE2EDuration="37.44095781s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:42:00.549821218 +0000 UTC m=+2.973591459" lastFinishedPulling="2026-06-02 04:42:33.015939454 +0000 UTC m=+35.439709697" observedRunningTime="2026-06-02 04:42:35.440056553 +0000 UTC m=+37.863826817" watchObservedRunningTime="2026-06-02 04:42:35.44095781 +0000 UTC m=+37.864728072" Jun 02 04:42:35.443776 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.443750 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl"] Jun 02 04:42:35.448282 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:35.448126 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c2faa7_ddc4_407f_806f_071afd1ef8c7.slice/crio-eea31d849cbaed3904e4305ac8af46cd32edc46167e7984a13a451a2a02d37b0 WatchSource:0}: Error finding container eea31d849cbaed3904e4305ac8af46cd32edc46167e7984a13a451a2a02d37b0: Status 404 returned error can't find the container with id eea31d849cbaed3904e4305ac8af46cd32edc46167e7984a13a451a2a02d37b0 Jun 02 04:42:35.508194 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.508169 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4v4x_d85df90f-0704-466b-bfc3-be341c63a82a/dns-node-resolver/0.log" Jun 02 04:42:35.869998 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.869967 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-bdsws"] Jun 02 04:42:35.898478 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.898447 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-bdsws"] Jun 02 04:42:35.898621 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.898570 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:35.901345 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.901322 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 02 04:42:35.901476 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.901424 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 02 04:42:35.901476 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.901423 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-c44mz\"" Jun 02 04:42:35.901588 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.901507 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 02 04:42:35.901685 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:35.901672 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 02 04:42:36.011046 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.011000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2x24\" (UniqueName: \"kubernetes.io/projected/928e05e0-aaf1-4708-92a4-ffedafd52a45-kube-api-access-s2x24\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.011046 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.011042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/928e05e0-aaf1-4708-92a4-ffedafd52a45-signing-key\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.011318 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.011112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/928e05e0-aaf1-4708-92a4-ffedafd52a45-signing-cabundle\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.112491 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.112450 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/928e05e0-aaf1-4708-92a4-ffedafd52a45-signing-cabundle\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.112650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.112530 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2x24\" (UniqueName: \"kubernetes.io/projected/928e05e0-aaf1-4708-92a4-ffedafd52a45-kube-api-access-s2x24\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.112650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.112559 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/928e05e0-aaf1-4708-92a4-ffedafd52a45-signing-key\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.113357 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.113338 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/928e05e0-aaf1-4708-92a4-ffedafd52a45-signing-cabundle\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.127311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.127241 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/928e05e0-aaf1-4708-92a4-ffedafd52a45-signing-key\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.127507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.127489 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2x24\" (UniqueName: \"kubernetes.io/projected/928e05e0-aaf1-4708-92a4-ffedafd52a45-kube-api-access-s2x24\") pod \"service-ca-685df94d57-bdsws\" (UID: \"928e05e0-aaf1-4708-92a4-ffedafd52a45\") " pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.209558 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.209518 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-bdsws" Jun 02 04:42:36.314752 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.314718 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:36.314887 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.314827 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:36.314887 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:36.314864 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 04:42:36.314974 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:36.314904 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 04:42:36.314974 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:36.314934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls podName:8a5ad758-ecd6-46a7-88d2-d17354c1b7d5 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:40.314915818 +0000 UTC m=+42.738686076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls") pod "dns-default-sjg59" (UID: "8a5ad758-ecd6-46a7-88d2-d17354c1b7d5") : secret "dns-default-metrics-tls" not found Jun 02 04:42:36.314974 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:36.314949 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert podName:16f61d36-7264-4685-811e-9bdfa53612e8 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:40.314942905 +0000 UTC m=+42.738713146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert") pod "ingress-canary-czq5w" (UID: "16f61d36-7264-4685-811e-9bdfa53612e8") : secret "canary-serving-cert" not found Jun 02 04:42:36.342539 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.342509 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-bdsws"] Jun 02 04:42:36.345814 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:36.345786 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928e05e0_aaf1_4708_92a4_ffedafd52a45.slice/crio-0c1bf506199b3f1d43f4a166819eb68b075f24d75dbd4fa18d0524dbfc8c0a31 WatchSource:0}: Error finding container 0c1bf506199b3f1d43f4a166819eb68b075f24d75dbd4fa18d0524dbfc8c0a31: Status 404 returned error can't find the container with id 0c1bf506199b3f1d43f4a166819eb68b075f24d75dbd4fa18d0524dbfc8c0a31 Jun 02 04:42:36.412038 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.411944 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-bdsws" event={"ID":"928e05e0-aaf1-4708-92a4-ffedafd52a45","Type":"ContainerStarted","Data":"0c1bf506199b3f1d43f4a166819eb68b075f24d75dbd4fa18d0524dbfc8c0a31"} Jun 02 04:42:36.413047 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.413022 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" event={"ID":"e4c2faa7-ddc4-407f-806f-071afd1ef8c7","Type":"ContainerStarted","Data":"eea31d849cbaed3904e4305ac8af46cd32edc46167e7984a13a451a2a02d37b0"} Jun 02 04:42:36.507073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:36.507044 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ndsfj_34e12dd2-625a-412d-b26c-206d4f40b16e/node-ca/0.log" Jun 02 04:42:38.418982 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:38.418881 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" event={"ID":"e4c2faa7-ddc4-407f-806f-071afd1ef8c7","Type":"ContainerStarted","Data":"844e81fd5ec0bdda6dda67cdd618d05c35f63afbbff0391be328dea98a05f339"} Jun 02 04:42:38.418982 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:38.418923 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" event={"ID":"e4c2faa7-ddc4-407f-806f-071afd1ef8c7","Type":"ContainerStarted","Data":"5cb17e9219533dfb90fd1a2295ab738cc0880096a9dc1b2aee13e67b84154f11"} Jun 02 04:42:38.420327 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:38.420305 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-bdsws" event={"ID":"928e05e0-aaf1-4708-92a4-ffedafd52a45","Type":"ContainerStarted","Data":"2dc9f6cd8979b1d5de698018786bb03b15383756584bb56958776fe8788c8ec6"} Jun 02 04:42:38.438315 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:38.438263 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-5rkhl" podStartSLOduration=2.437889064 podStartE2EDuration="4.43824926s" podCreationTimestamp="2026-06-02 04:42:34 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.450085049 +0000 UTC m=+37.873855308" lastFinishedPulling="2026-06-02 04:42:37.45044526 +0000 UTC m=+39.874215504" observedRunningTime="2026-06-02 04:42:38.438027362 +0000 UTC m=+40.861797626" watchObservedRunningTime="2026-06-02 04:42:38.43824926 +0000 UTC m=+40.862019522" Jun 02 04:42:38.458551 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:38.458500 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-bdsws" podStartSLOduration=1.64389505 podStartE2EDuration="3.458484912s" podCreationTimestamp="2026-06-02 04:42:35 +0000 UTC" firstStartedPulling="2026-06-02 04:42:36.350129738 +0000 UTC m=+38.773899979" lastFinishedPulling="2026-06-02 04:42:38.164719589 +0000 UTC m=+40.588489841" observedRunningTime="2026-06-02 04:42:38.458233152 +0000 UTC m=+40.882003440" watchObservedRunningTime="2026-06-02 04:42:38.458484912 +0000 UTC m=+40.882255179" Jun 02 04:42:40.350131 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:40.350087 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:40.350630 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:40.350162 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:40.350630 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:40.350270 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 04:42:40.350630 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:40.350289 2566 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 04:42:40.350630 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:40.350336 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls podName:8a5ad758-ecd6-46a7-88d2-d17354c1b7d5 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:48.350318578 +0000 UTC m=+50.774088833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls") pod "dns-default-sjg59" (UID: "8a5ad758-ecd6-46a7-88d2-d17354c1b7d5") : secret "dns-default-metrics-tls" not found Jun 02 04:42:40.350630 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:42:40.350358 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert podName:16f61d36-7264-4685-811e-9bdfa53612e8 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:48.350346088 +0000 UTC m=+50.774116334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert") pod "ingress-canary-czq5w" (UID: "16f61d36-7264-4685-811e-9bdfa53612e8") : secret "canary-serving-cert" not found Jun 02 04:42:48.405113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.405066 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:48.405113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.405121 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:48.407653 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.407624 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a5ad758-ecd6-46a7-88d2-d17354c1b7d5-metrics-tls\") pod \"dns-default-sjg59\" (UID: \"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5\") " pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:48.407789 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.407752 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16f61d36-7264-4685-811e-9bdfa53612e8-cert\") pod \"ingress-canary-czq5w\" (UID: \"16f61d36-7264-4685-811e-9bdfa53612e8\") " pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:48.473297 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.473254 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:48.488093 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.488066 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-czq5w" Jun 02 04:42:48.608212 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.608114 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sjg59"] Jun 02 04:42:48.612254 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:48.612228 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5ad758_ecd6_46a7_88d2_d17354c1b7d5.slice/crio-d7480464a8ee1828aa23e67dafedcf538136921ef17fe230bd1bad55e849b9cd WatchSource:0}: Error finding container d7480464a8ee1828aa23e67dafedcf538136921ef17fe230bd1bad55e849b9cd: Status 404 returned error can't find the container with id d7480464a8ee1828aa23e67dafedcf538136921ef17fe230bd1bad55e849b9cd Jun 02 04:42:48.625666 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:48.625639 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-czq5w"] Jun 02 04:42:48.629479 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:48.629453 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f61d36_7264_4685_811e_9bdfa53612e8.slice/crio-a79279b0dd96e9c38edfd71408cc4b0fea10de88ccdd10e1ac7bba5ada1cacda WatchSource:0}: Error finding container a79279b0dd96e9c38edfd71408cc4b0fea10de88ccdd10e1ac7bba5ada1cacda: Status 404 returned error can't find the container with id a79279b0dd96e9c38edfd71408cc4b0fea10de88ccdd10e1ac7bba5ada1cacda Jun 02 04:42:49.444532 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:49.444469 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-czq5w" event={"ID":"16f61d36-7264-4685-811e-9bdfa53612e8","Type":"ContainerStarted","Data":"a79279b0dd96e9c38edfd71408cc4b0fea10de88ccdd10e1ac7bba5ada1cacda"} Jun 02 04:42:49.446306 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:49.446254 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjg59" event={"ID":"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5","Type":"ContainerStarted","Data":"d7480464a8ee1828aa23e67dafedcf538136921ef17fe230bd1bad55e849b9cd"} Jun 02 04:42:51.453175 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:51.453122 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-czq5w" event={"ID":"16f61d36-7264-4685-811e-9bdfa53612e8","Type":"ContainerStarted","Data":"e0da45bc9152f48ed02ac159882f378a2d71dfdf38089588b8284148fbffa487"} Jun 02 04:42:51.454693 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:51.454664 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjg59" event={"ID":"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5","Type":"ContainerStarted","Data":"f8b150f4fc58785dc31b963f2140983e3542492347e19240e29b8d3471228915"} Jun 02 04:42:51.454812 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:51.454702 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjg59" event={"ID":"8a5ad758-ecd6-46a7-88d2-d17354c1b7d5","Type":"ContainerStarted","Data":"7522521b5c76979e2fcdf8c831eeefacabfa726c1c983746a919631176b66f3f"} Jun 02 04:42:51.454853 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:51.454813 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sjg59" Jun 02 04:42:51.469893 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:51.469801 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-czq5w" podStartSLOduration=17.122986746 podStartE2EDuration="19.469788818s" podCreationTimestamp="2026-06-02 04:42:32 +0000 UTC" firstStartedPulling="2026-06-02 04:42:48.631810686 +0000 UTC m=+51.055580926" lastFinishedPulling="2026-06-02 04:42:50.978612744 +0000 UTC m=+53.402382998" observedRunningTime="2026-06-02 04:42:51.469764789 +0000 UTC m=+53.893535049" watchObservedRunningTime="2026-06-02 04:42:51.469788818 +0000 UTC m=+53.893559078" Jun 02 04:42:51.492625 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:51.492577 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-sjg59" podStartSLOduration=17.133452126 podStartE2EDuration="19.492562933s" podCreationTimestamp="2026-06-02 04:42:32 +0000 UTC" firstStartedPulling="2026-06-02 04:42:48.614556117 +0000 UTC m=+51.038326372" lastFinishedPulling="2026-06-02 04:42:50.973666934 +0000 UTC m=+53.397437179" observedRunningTime="2026-06-02 04:42:51.491723431 +0000 UTC m=+53.915493693" watchObservedRunningTime="2026-06-02 04:42:51.492562933 +0000 UTC m=+53.916333196" Jun 02 04:42:55.955010 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.954974 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qdb2w"] Jun 02 04:42:55.962987 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.962962 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:55.966061 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.966037 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 04:42:55.966189 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.966087 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v6k97\"" Jun 02 04:42:55.966888 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.966869 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 04:42:55.966976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.966889 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 04:42:55.966976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.966870 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 04:42:55.972687 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:55.972667 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qdb2w"] Jun 02 04:42:56.056263 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.056226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.056447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.056284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-crio-socket\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.056447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.056346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-data-volume\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.056447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.056385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.056447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.056439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78l57\" (UniqueName: \"kubernetes.io/projected/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-kube-api-access-78l57\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157369 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157333 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-data-volume\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157369 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157371 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157415 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-78l57\" (UniqueName: \"kubernetes.io/projected/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-kube-api-access-78l57\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157438 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157465 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-crio-socket\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157550 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-crio-socket\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.157869 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.157666 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-data-volume\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.158075 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.158053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.159667 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.159652 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.181588 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.181551 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66dc564ff9-6fcj8"] Jun 02 04:42:56.183640 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.183618 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.186424 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.186404 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78l57\" (UniqueName: \"kubernetes.io/projected/d58f9f19-c5d4-41a2-a6b9-9421e39aeca4-kube-api-access-78l57\") pod \"insights-runtime-extractor-qdb2w\" (UID: \"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4\") " pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.186722 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.186708 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 04:42:56.187045 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.187027 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8jbvs\"" Jun 02 04:42:56.187085 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.187027 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 04:42:56.187363 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.187346 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 04:42:56.192554 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.192532 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 04:42:56.199817 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.199798 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66dc564ff9-6fcj8"] Jun 02 04:42:56.258130 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34b27fbd-93e6-432d-9fac-6a7cb7642226-installation-pull-secrets\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258130 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258089 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34b27fbd-93e6-432d-9fac-6a7cb7642226-registry-certificates\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258130 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258106 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f4sw\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-kube-api-access-5f4sw\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258355 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34b27fbd-93e6-432d-9fac-6a7cb7642226-image-registry-private-configuration\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258355 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258230 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-registry-tls\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258355 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-bound-sa-token\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258355 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34b27fbd-93e6-432d-9fac-6a7cb7642226-ca-trust-extracted\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.258355 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.258327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34b27fbd-93e6-432d-9fac-6a7cb7642226-trusted-ca\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.272135 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.272110 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qdb2w" Jun 02 04:42:56.358832 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358795 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34b27fbd-93e6-432d-9fac-6a7cb7642226-registry-certificates\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.358832 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358838 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5f4sw\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-kube-api-access-5f4sw\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359069 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358873 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34b27fbd-93e6-432d-9fac-6a7cb7642226-image-registry-private-configuration\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359069 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358894 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-registry-tls\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359069 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358917 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-bound-sa-token\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359069 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358949 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34b27fbd-93e6-432d-9fac-6a7cb7642226-ca-trust-extracted\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359069 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.358997 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34b27fbd-93e6-432d-9fac-6a7cb7642226-trusted-ca\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359069 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.359033 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34b27fbd-93e6-432d-9fac-6a7cb7642226-installation-pull-secrets\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359779 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.359720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34b27fbd-93e6-432d-9fac-6a7cb7642226-registry-certificates\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.359878 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.359829 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34b27fbd-93e6-432d-9fac-6a7cb7642226-ca-trust-extracted\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.360327 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.360301 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34b27fbd-93e6-432d-9fac-6a7cb7642226-trusted-ca\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.362055 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.362006 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-registry-tls\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.362197 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.362178 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34b27fbd-93e6-432d-9fac-6a7cb7642226-image-registry-private-configuration\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.362539 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.362516 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34b27fbd-93e6-432d-9fac-6a7cb7642226-installation-pull-secrets\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.372212 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.372186 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f4sw\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-kube-api-access-5f4sw\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.372471 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.372453 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34b27fbd-93e6-432d-9fac-6a7cb7642226-bound-sa-token\") pod \"image-registry-66dc564ff9-6fcj8\" (UID: \"34b27fbd-93e6-432d-9fac-6a7cb7642226\") " pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.392541 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.392510 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hjcp9" Jun 02 04:42:56.413848 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.413810 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qdb2w"] Jun 02 04:42:56.418231 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:56.418197 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58f9f19_c5d4_41a2_a6b9_9421e39aeca4.slice/crio-bcf5566ed1e72338bc3d6bf3258a1085730c05a27fafe7919d8b471e1b08f646 WatchSource:0}: Error finding container bcf5566ed1e72338bc3d6bf3258a1085730c05a27fafe7919d8b471e1b08f646: Status 404 returned error can't find the container with id bcf5566ed1e72338bc3d6bf3258a1085730c05a27fafe7919d8b471e1b08f646 Jun 02 04:42:56.468964 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.468926 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdb2w" event={"ID":"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4","Type":"ContainerStarted","Data":"bcf5566ed1e72338bc3d6bf3258a1085730c05a27fafe7919d8b471e1b08f646"} Jun 02 04:42:56.498961 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.498921 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:56.631643 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:56.631608 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66dc564ff9-6fcj8"] Jun 02 04:42:56.635156 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:42:56.635113 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b27fbd_93e6_432d_9fac_6a7cb7642226.slice/crio-d2888a2f9d178af6458b4e70a0ccf8507c519750efe8c144b50f01b98d893738 WatchSource:0}: Error finding container d2888a2f9d178af6458b4e70a0ccf8507c519750efe8c144b50f01b98d893738: Status 404 returned error can't find the container with id d2888a2f9d178af6458b4e70a0ccf8507c519750efe8c144b50f01b98d893738 Jun 02 04:42:57.473228 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:57.473192 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdb2w" event={"ID":"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4","Type":"ContainerStarted","Data":"408185a8bbaa66ffad6812ca002576797f6ad824b105906ab1eac9824a4812a6"} Jun 02 04:42:57.473228 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:57.473233 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdb2w" event={"ID":"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4","Type":"ContainerStarted","Data":"4c208bf008f34234b3224c8f0829b6aae2e64bdb9ed38ed42dc62f36af590145"} Jun 02 04:42:57.474413 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:57.474389 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" event={"ID":"34b27fbd-93e6-432d-9fac-6a7cb7642226","Type":"ContainerStarted","Data":"34228d608ef25f3a53139ad728731b8a46694cfce9ab8bc60561e71af7d48dd4"} Jun 02 04:42:57.474413 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:57.474416 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" event={"ID":"34b27fbd-93e6-432d-9fac-6a7cb7642226","Type":"ContainerStarted","Data":"d2888a2f9d178af6458b4e70a0ccf8507c519750efe8c144b50f01b98d893738"} Jun 02 04:42:57.474662 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:57.474644 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:42:57.496220 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:57.496172 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" podStartSLOduration=1.4961595189999999 podStartE2EDuration="1.496159519s" podCreationTimestamp="2026-06-02 04:42:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:42:57.495084206 +0000 UTC m=+59.918854466" watchObservedRunningTime="2026-06-02 04:42:57.496159519 +0000 UTC m=+59.919929779" Jun 02 04:42:59.481402 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:59.481361 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdb2w" event={"ID":"d58f9f19-c5d4-41a2-a6b9-9421e39aeca4","Type":"ContainerStarted","Data":"98b44bb28f244a105ca3ee47104dac50ac798ec51356a90c325a77bc49a19997"} Jun 02 04:42:59.500432 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:42:59.500383 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qdb2w" podStartSLOduration=2.122096304 podStartE2EDuration="4.500369746s" podCreationTimestamp="2026-06-02 04:42:55 +0000 UTC" firstStartedPulling="2026-06-02 04:42:56.475048548 +0000 UTC m=+58.898818789" lastFinishedPulling="2026-06-02 04:42:58.853321986 +0000 UTC m=+61.277092231" observedRunningTime="2026-06-02 04:42:59.499296088 +0000 UTC m=+61.923066350" watchObservedRunningTime="2026-06-02 04:42:59.500369746 +0000 UTC m=+61.924140009" Jun 02 04:43:01.458725 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:01.458690 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sjg59" Jun 02 04:43:03.812820 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.812775 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:43:03.815533 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.815508 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 04:43:03.825829 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.825805 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05f924c-4cf3-4db7-bb80-baa2e515793e-metrics-certs\") pod \"network-metrics-daemon-l8f45\" (UID: \"b05f924c-4cf3-4db7-bb80-baa2e515793e\") " pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:43:03.914149 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.914111 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:43:03.916984 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.916965 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 04:43:03.927032 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.927009 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 04:43:03.938374 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:03.938351 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-846t7\" (UniqueName: \"kubernetes.io/projected/6a73057d-6f6b-4e14-97a7-a6e755a2cfa0-kube-api-access-846t7\") pod \"network-check-target-qpm4h\" (UID: \"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0\") " pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:43:04.025969 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.025940 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c2frh\"" Jun 02 04:43:04.033927 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.033901 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l8f45" Jun 02 04:43:04.126290 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.126260 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rvl8h\"" Jun 02 04:43:04.134469 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.134437 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:43:04.175948 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.175893 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l8f45"] Jun 02 04:43:04.179060 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:04.179019 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05f924c_4cf3_4db7_bb80_baa2e515793e.slice/crio-b5797a370cf647b3206d55a263f8f2bf42c17cc9b34c1468b59aa44fbd39577e WatchSource:0}: Error finding container b5797a370cf647b3206d55a263f8f2bf42c17cc9b34c1468b59aa44fbd39577e: Status 404 returned error can't find the container with id b5797a370cf647b3206d55a263f8f2bf42c17cc9b34c1468b59aa44fbd39577e Jun 02 04:43:04.262967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.262935 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qpm4h"] Jun 02 04:43:04.265793 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:04.265769 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a73057d_6f6b_4e14_97a7_a6e755a2cfa0.slice/crio-19a570349e071235def5ab66727be9ae201a98209edde0decfedba8dc017b27e WatchSource:0}: Error finding container 19a570349e071235def5ab66727be9ae201a98209edde0decfedba8dc017b27e: Status 404 returned error can't find the container with id 19a570349e071235def5ab66727be9ae201a98209edde0decfedba8dc017b27e Jun 02 04:43:04.413169 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.413068 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c5c7db546-rkz7v"] Jun 02 04:43:04.415615 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.415597 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.418410 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418386 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mbdsv\"" Jun 02 04:43:04.418636 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418609 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 04:43:04.418713 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418671 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 04:43:04.418963 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418925 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 04:43:04.418963 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418939 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 04:43:04.418963 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418957 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 04:43:04.419173 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.418978 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 04:43:04.419173 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.419032 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 04:43:04.429525 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.429502 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c5c7db546-rkz7v"] Jun 02 04:43:04.497284 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.497247 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qpm4h" event={"ID":"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0","Type":"ContainerStarted","Data":"19a570349e071235def5ab66727be9ae201a98209edde0decfedba8dc017b27e"} Jun 02 04:43:04.498195 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.498164 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l8f45" event={"ID":"b05f924c-4cf3-4db7-bb80-baa2e515793e","Type":"ContainerStarted","Data":"b5797a370cf647b3206d55a263f8f2bf42c17cc9b34c1468b59aa44fbd39577e"} Jun 02 04:43:04.518677 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.518647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-config\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.518802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.518683 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-service-ca\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.518802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.518699 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-oauth-serving-cert\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.518802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.518726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-oauth-config\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.518802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.518755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-serving-cert\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.518802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.518779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2cwk\" (UniqueName: \"kubernetes.io/projected/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-kube-api-access-m2cwk\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.619516 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.619483 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-serving-cert\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.619711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.619533 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m2cwk\" (UniqueName: \"kubernetes.io/projected/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-kube-api-access-m2cwk\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.619711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.619592 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-config\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.619711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.619623 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-service-ca\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.619711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.619646 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-oauth-serving-cert\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.619711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.619678 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-oauth-config\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.620416 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.620388 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-service-ca\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.620416 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.620401 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-config\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.620580 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.620490 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-oauth-serving-cert\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.622366 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.622345 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-oauth-config\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.622461 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.622378 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-serving-cert\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.632704 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.632680 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2cwk\" (UniqueName: \"kubernetes.io/projected/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-kube-api-access-m2cwk\") pod \"console-6c5c7db546-rkz7v\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.724826 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.724734 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:04.887185 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:04.887149 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c5c7db546-rkz7v"] Jun 02 04:43:04.890624 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:04.890589 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef7102c_dd31_41df_8aee_27f0b02d6fb2.slice/crio-7516eea6c16569e645df36e8fa937de7b6bdc85b97cf2a510d726249922e442c WatchSource:0}: Error finding container 7516eea6c16569e645df36e8fa937de7b6bdc85b97cf2a510d726249922e442c: Status 404 returned error can't find the container with id 7516eea6c16569e645df36e8fa937de7b6bdc85b97cf2a510d726249922e442c Jun 02 04:43:05.506222 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:05.505482 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l8f45" event={"ID":"b05f924c-4cf3-4db7-bb80-baa2e515793e","Type":"ContainerStarted","Data":"1d2d426c8d6addbf7061a191abb5852ee52d125fbc64ba991b502c995b6211a7"} Jun 02 04:43:05.506222 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:05.505523 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l8f45" event={"ID":"b05f924c-4cf3-4db7-bb80-baa2e515793e","Type":"ContainerStarted","Data":"e8d515b3c455f9dbd5af788e20ff2c5b097f30da804e1698b08b97116d795e2b"} Jun 02 04:43:05.507379 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:05.507346 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5c7db546-rkz7v" event={"ID":"8ef7102c-dd31-41df-8aee-27f0b02d6fb2","Type":"ContainerStarted","Data":"7516eea6c16569e645df36e8fa937de7b6bdc85b97cf2a510d726249922e442c"} Jun 02 04:43:05.534227 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:05.534160 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l8f45" podStartSLOduration=66.426299101 podStartE2EDuration="1m7.534126084s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:43:04.181091904 +0000 UTC m=+66.604862159" lastFinishedPulling="2026-06-02 04:43:05.288918887 +0000 UTC m=+67.712689142" observedRunningTime="2026-06-02 04:43:05.532018744 +0000 UTC m=+67.955788997" watchObservedRunningTime="2026-06-02 04:43:05.534126084 +0000 UTC m=+67.957896342" Jun 02 04:43:06.498752 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.498315 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-phw6w"] Jun 02 04:43:06.504062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.504036 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.507542 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.507088 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 04:43:06.507542 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.507406 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 04:43:06.508610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.508192 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 04:43:06.508610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.508443 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 04:43:06.508610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.508501 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 04:43:06.508828 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.508748 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 04:43:06.509157 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.508981 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jmbvm\"" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-sys\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-textfile\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-root\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536624 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-accelerators-collector-config\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-metrics-client-ca\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jvd2\" (UniqueName: \"kubernetes.io/projected/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-kube-api-access-5jvd2\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-tls\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.537068 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.536759 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-wtmp\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637430 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-root\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637491 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637514 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-accelerators-collector-config\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637558 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-metrics-client-ca\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637585 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5jvd2\" (UniqueName: \"kubernetes.io/projected/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-kube-api-access-5jvd2\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637615 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-tls\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637633 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-wtmp\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637653 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-sys\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.637988 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.637672 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-textfile\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.638655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.638040 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-textfile\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.638655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.638250 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-wtmp\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.638655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.638418 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-sys\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.638655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.638454 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-root\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.638655 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.638605 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-metrics-client-ca\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.638957 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.638906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-accelerators-collector-config\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.641058 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.641017 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-tls\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.641832 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.641793 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.652846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.652798 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jvd2\" (UniqueName: \"kubernetes.io/projected/ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42-kube-api-access-5jvd2\") pod \"node-exporter-phw6w\" (UID: \"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42\") " pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:06.820055 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:06.820014 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-phw6w" Jun 02 04:43:07.517895 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.517855 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:43:07.538456 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.538419 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.558765 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.558589 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 04:43:07.558765 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.558589 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6db8r\"" Jun 02 04:43:07.558765 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.558622 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 04:43:07.558765 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.558644 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 04:43:07.558765 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.558721 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 04:43:07.559085 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.558961 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 04:43:07.559085 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.559060 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 04:43:07.559205 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.559187 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 04:43:07.559373 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.559324 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 04:43:07.559487 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.559404 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 04:43:07.608057 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.608015 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:43:07.647824 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.647791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-out\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.647994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.647838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.647994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.647863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-volume\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.647994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.647886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.647994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.647980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648061 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45thl\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-kube-api-access-45thl\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648132 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-web-config\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.648275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.648251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749678 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749635 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749687 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-volume\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749729 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749766 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749791 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749819 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45thl\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-kube-api-access-45thl\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.749855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749854 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749886 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749910 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749933 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-web-config\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749959 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.749985 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750507 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.750030 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-out\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.750859 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.750831 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.751666 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.751640 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.753120 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.753003 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-volume\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.753120 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:07.753108 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 02 04:43:07.753281 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:07.753195 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls podName:748ef5cb-4cfe-485d-9090-3a5bdc407c5e nodeName:}" failed. No retries permitted until 2026-06-02 04:43:08.25317535 +0000 UTC m=+70.676945599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e") : secret "alertmanager-main-tls" not found Jun 02 04:43:07.754048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.753659 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.754048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.754007 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.754277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.754254 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.754461 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.754437 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.754634 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.754610 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-web-config\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.754887 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.754784 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.755041 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.755016 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-out\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.756757 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.756732 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:07.759683 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:07.759663 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45thl\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-kube-api-access-45thl\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:08.254643 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:08.254605 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:08.257466 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:08.257432 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:08.451619 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:08.451578 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:43:09.271707 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.253460 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:43:09.271707 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:09.253957 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748ef5cb_4cfe_485d_9090_3a5bdc407c5e.slice/crio-605acab68b01e589208f4fc8ea5f5268885ad3fd40e9a177c909923aca60e464 WatchSource:0}: Error finding container 605acab68b01e589208f4fc8ea5f5268885ad3fd40e9a177c909923aca60e464: Status 404 returned error can't find the container with id 605acab68b01e589208f4fc8ea5f5268885ad3fd40e9a177c909923aca60e464 Jun 02 04:43:09.526596 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.526340 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"605acab68b01e589208f4fc8ea5f5268885ad3fd40e9a177c909923aca60e464"} Jun 02 04:43:09.528023 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.527960 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5c7db546-rkz7v" event={"ID":"8ef7102c-dd31-41df-8aee-27f0b02d6fb2","Type":"ContainerStarted","Data":"0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826"} Jun 02 04:43:09.530184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.529798 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qpm4h" event={"ID":"6a73057d-6f6b-4e14-97a7-a6e755a2cfa0","Type":"ContainerStarted","Data":"270604cce4f74b3efc12f6c2a2b9891bc532577ca5106fa380dd40f1c59f8bc0"} Jun 02 04:43:09.530184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.529926 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:43:09.531479 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.531426 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phw6w" event={"ID":"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42","Type":"ContainerStarted","Data":"f007fc14914fe9c0370fd64742aa92b83aa9228f89f410f6deb98af82437bd81"} Jun 02 04:43:09.551817 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.551759 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6c5c7db546-rkz7v" podStartSLOduration=1.336541477 podStartE2EDuration="5.551738829s" podCreationTimestamp="2026-06-02 04:43:04 +0000 UTC" firstStartedPulling="2026-06-02 04:43:04.892911197 +0000 UTC m=+67.316681439" lastFinishedPulling="2026-06-02 04:43:09.108108546 +0000 UTC m=+71.531878791" observedRunningTime="2026-06-02 04:43:09.55028992 +0000 UTC m=+71.974060194" watchObservedRunningTime="2026-06-02 04:43:09.551738829 +0000 UTC m=+71.975509093" Jun 02 04:43:09.574520 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:09.574456 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qpm4h" podStartSLOduration=66.73396389 podStartE2EDuration="1m11.574440165s" podCreationTimestamp="2026-06-02 04:41:58 +0000 UTC" firstStartedPulling="2026-06-02 04:43:04.267624908 +0000 UTC m=+66.691395152" lastFinishedPulling="2026-06-02 04:43:09.108101168 +0000 UTC m=+71.531871427" observedRunningTime="2026-06-02 04:43:09.572527521 +0000 UTC m=+71.996297785" watchObservedRunningTime="2026-06-02 04:43:09.574440165 +0000 UTC m=+71.998210427" Jun 02 04:43:10.536560 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:10.536524 2566 generic.go:358] "Generic (PLEG): container finished" podID="ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42" containerID="f164eaaa227446284a2da18175a5abe747e4ec578f4721141cbf8280117e0bdb" exitCode=0 Jun 02 04:43:10.537055 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:10.536618 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phw6w" event={"ID":"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42","Type":"ContainerDied","Data":"f164eaaa227446284a2da18175a5abe747e4ec578f4721141cbf8280117e0bdb"} Jun 02 04:43:11.199002 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.198968 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785"] Jun 02 04:43:11.200993 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.200973 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:11.203325 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.203300 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 04:43:11.203414 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.203366 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-tlbgn\"" Jun 02 04:43:11.212957 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.212932 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785"] Jun 02 04:43:11.283297 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.283245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-t6785\" (UID: \"e5c2750a-a1f1-4747-8781-95d65cc299f8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:11.384599 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.384563 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-t6785\" (UID: \"e5c2750a-a1f1-4747-8781-95d65cc299f8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:11.384743 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:11.384727 2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 02 04:43:11.384808 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:11.384797 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert podName:e5c2750a-a1f1-4747-8781-95d65cc299f8 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:11.884778639 +0000 UTC m=+74.308548883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-t6785" (UID: "e5c2750a-a1f1-4747-8781-95d65cc299f8") : secret "monitoring-plugin-cert" not found Jun 02 04:43:11.541454 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.541415 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phw6w" event={"ID":"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42","Type":"ContainerStarted","Data":"ffa116db5fd36999772597f421a279b52d772608fa5ab26b32c56d0764be580f"} Jun 02 04:43:11.541454 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.541459 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phw6w" event={"ID":"ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42","Type":"ContainerStarted","Data":"26045cbe1a944e9c8498bc7f1a3360914562dc15c88691fca3c41c3dd91c220e"} Jun 02 04:43:11.542978 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.542949 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626" exitCode=0 Jun 02 04:43:11.543080 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.543034 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626"} Jun 02 04:43:11.561883 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.561833 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-phw6w" podStartSLOduration=4.760895151 podStartE2EDuration="5.56181932s" podCreationTimestamp="2026-06-02 04:43:06 +0000 UTC" firstStartedPulling="2026-06-02 04:43:09.108357565 +0000 UTC m=+71.532127808" lastFinishedPulling="2026-06-02 04:43:09.909281723 +0000 UTC m=+72.333051977" observedRunningTime="2026-06-02 04:43:11.560571202 +0000 UTC m=+73.984341478" watchObservedRunningTime="2026-06-02 04:43:11.56181932 +0000 UTC m=+73.985589618" Jun 02 04:43:11.707848 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.707760 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78f698bbc-62kfd"] Jun 02 04:43:11.710401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.710380 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.714294 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.714273 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 02 04:43:11.714416 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.714297 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 02 04:43:11.714416 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.714273 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 02 04:43:11.714571 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.714558 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 02 04:43:11.715217 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.715201 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 02 04:43:11.720616 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.720600 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qphd5\"" Jun 02 04:43:11.728801 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.728775 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78f698bbc-62kfd"] Jun 02 04:43:11.737047 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.737026 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 02 04:43:11.789151 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-serving-certs-ca-bundle\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789305 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789305 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-telemeter-client-tls\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789305 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-federate-client-tls\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789462 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789462 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8zfb\" (UniqueName: \"kubernetes.io/projected/f8571773-8d33-445c-9783-dab4aa2eaee4-kube-api-access-d8zfb\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789462 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-secret-telemeter-client\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.789578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.789488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-metrics-client-ca\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.890547 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890503 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.890547 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890553 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-t6785\" (UID: \"e5c2750a-a1f1-4747-8781-95d65cc299f8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:11.890814 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890573 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8zfb\" (UniqueName: \"kubernetes.io/projected/f8571773-8d33-445c-9783-dab4aa2eaee4-kube-api-access-d8zfb\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.890814 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890616 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-secret-telemeter-client\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.890814 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890641 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-metrics-client-ca\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.890814 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:11.890670 2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 02 04:43:11.890814 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:11.890749 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert podName:e5c2750a-a1f1-4747-8781-95d65cc299f8 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:12.890728322 +0000 UTC m=+75.314498563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-t6785" (UID: "e5c2750a-a1f1-4747-8781-95d65cc299f8") : secret "monitoring-plugin-cert" not found Jun 02 04:43:11.890814 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890674 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-serving-certs-ca-bundle\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.891099 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890831 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.891099 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890862 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-telemeter-client-tls\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.891099 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.890897 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-federate-client-tls\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.892037 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.891980 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-serving-certs-ca-bundle\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.892167 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.892090 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-metrics-client-ca\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.892283 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.892258 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8571773-8d33-445c-9783-dab4aa2eaee4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.893601 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.893577 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.893702 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.893682 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-secret-telemeter-client\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.893769 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.893752 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-federate-client-tls\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.893805 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.893775 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f8571773-8d33-445c-9783-dab4aa2eaee4-telemeter-client-tls\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:11.898879 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:11.898859 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8zfb\" (UniqueName: \"kubernetes.io/projected/f8571773-8d33-445c-9783-dab4aa2eaee4-kube-api-access-d8zfb\") pod \"telemeter-client-78f698bbc-62kfd\" (UID: \"f8571773-8d33-445c-9783-dab4aa2eaee4\") " pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:12.021786 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:12.021688 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" Jun 02 04:43:12.173352 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:12.173321 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78f698bbc-62kfd"] Jun 02 04:43:12.176092 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:12.176058 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8571773_8d33_445c_9783_dab4aa2eaee4.slice/crio-5589ca212cc4071c264589f5023a9029914560a58660246bb1cb0e27f12e1a4c WatchSource:0}: Error finding container 5589ca212cc4071c264589f5023a9029914560a58660246bb1cb0e27f12e1a4c: Status 404 returned error can't find the container with id 5589ca212cc4071c264589f5023a9029914560a58660246bb1cb0e27f12e1a4c Jun 02 04:43:12.549095 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:12.549052 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" event={"ID":"f8571773-8d33-445c-9783-dab4aa2eaee4","Type":"ContainerStarted","Data":"5589ca212cc4071c264589f5023a9029914560a58660246bb1cb0e27f12e1a4c"} Jun 02 04:43:12.900682 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:12.900649 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-t6785\" (UID: \"e5c2750a-a1f1-4747-8781-95d65cc299f8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:12.903598 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:12.903569 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5c2750a-a1f1-4747-8781-95d65cc299f8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-t6785\" (UID: \"e5c2750a-a1f1-4747-8781-95d65cc299f8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:13.010931 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.010890 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:13.226807 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.226433 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785"] Jun 02 04:43:13.553975 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.553943 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d"} Jun 02 04:43:13.553975 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.553977 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3"} Jun 02 04:43:13.554438 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.553987 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2"} Jun 02 04:43:13.554438 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.554004 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4"} Jun 02 04:43:13.554438 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.554013 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e"} Jun 02 04:43:13.554969 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:13.554945 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" event={"ID":"e5c2750a-a1f1-4747-8781-95d65cc299f8","Type":"ContainerStarted","Data":"485bb5096d219a08b21e549ebd7b6d1f0dfb60182630d58eb38d3f797770f8d0"} Jun 02 04:43:14.724951 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:14.724921 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:14.724951 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:14.724955 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:14.729962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:14.729934 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:14.989014 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:14.988920 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-76d68c6d7b-f6sxk"] Jun 02 04:43:14.995198 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:14.995168 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.002723 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.002690 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d68c6d7b-f6sxk"] Jun 02 04:43:15.002969 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.002936 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 04:43:15.121180 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-config\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.121265 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121223 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-serving-cert\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.121265 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-trusted-ca-bundle\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.121336 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-oauth-serving-cert\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.121336 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-service-ca\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.121401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44rq2\" (UniqueName: \"kubernetes.io/projected/b3312699-7284-4457-8d5f-7e6ff0e6732a-kube-api-access-44rq2\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.121401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.121382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-oauth-config\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.222800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.222760 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-serving-cert\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.222800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.222799 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-trusted-ca-bundle\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.222818 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-oauth-serving-cert\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.222834 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-service-ca\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.222944 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-44rq2\" (UniqueName: \"kubernetes.io/projected/b3312699-7284-4457-8d5f-7e6ff0e6732a-kube-api-access-44rq2\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.223013 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-oauth-config\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.223050 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-config\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223686 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.223661 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-service-ca\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223804 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.223682 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-oauth-serving-cert\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223804 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.223759 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-trusted-ca-bundle\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.223804 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.223763 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-config\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.225421 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.225392 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-oauth-config\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.225604 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.225584 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-serving-cert\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.231951 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.231932 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44rq2\" (UniqueName: \"kubernetes.io/projected/b3312699-7284-4457-8d5f-7e6ff0e6732a-kube-api-access-44rq2\") pod \"console-76d68c6d7b-f6sxk\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.307060 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.306964 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:15.435617 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.435579 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d68c6d7b-f6sxk"] Jun 02 04:43:15.439556 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:15.439527 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3312699_7284_4457_8d5f_7e6ff0e6732a.slice/crio-9c5c3ff9cdae783e491de41ce5d2d59ee5d41b7fb067b03e986880febb65c8f4 WatchSource:0}: Error finding container 9c5c3ff9cdae783e491de41ce5d2d59ee5d41b7fb067b03e986880febb65c8f4: Status 404 returned error can't find the container with id 9c5c3ff9cdae783e491de41ce5d2d59ee5d41b7fb067b03e986880febb65c8f4 Jun 02 04:43:15.567342 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.567300 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" event={"ID":"f8571773-8d33-445c-9783-dab4aa2eaee4","Type":"ContainerStarted","Data":"30a559022d4eeda566bd0858d5500da00b5e671d49e1233b4b2c1488665267b6"} Jun 02 04:43:15.567496 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.567348 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" event={"ID":"f8571773-8d33-445c-9783-dab4aa2eaee4","Type":"ContainerStarted","Data":"d0c90bb8af94dd4b7eda7ff0ce0f5350f4b39427b5867769b95807537e6fd560"} Jun 02 04:43:15.567496 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.567361 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" event={"ID":"f8571773-8d33-445c-9783-dab4aa2eaee4","Type":"ContainerStarted","Data":"6765fee750beaa6566aa8b40b0fad4a30d347e3e8fc6d50055e5256edbcd15d9"} Jun 02 04:43:15.568855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.568827 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d68c6d7b-f6sxk" event={"ID":"b3312699-7284-4457-8d5f-7e6ff0e6732a","Type":"ContainerStarted","Data":"0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb"} Jun 02 04:43:15.568855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.568858 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d68c6d7b-f6sxk" event={"ID":"b3312699-7284-4457-8d5f-7e6ff0e6732a","Type":"ContainerStarted","Data":"9c5c3ff9cdae783e491de41ce5d2d59ee5d41b7fb067b03e986880febb65c8f4"} Jun 02 04:43:15.572203 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.572175 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerStarted","Data":"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01"} Jun 02 04:43:15.573694 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.573669 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" event={"ID":"e5c2750a-a1f1-4747-8781-95d65cc299f8","Type":"ContainerStarted","Data":"e16a9d8ed029fa8898bf3fb19f37ec938b2aa417f69d2e761f2b4fdea251a96a"} Jun 02 04:43:15.573845 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.573823 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:15.577609 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.577586 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:15.577939 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.577922 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" Jun 02 04:43:15.588631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.588585 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78f698bbc-62kfd" podStartSLOduration=2.123743984 podStartE2EDuration="4.588574211s" podCreationTimestamp="2026-06-02 04:43:11 +0000 UTC" firstStartedPulling="2026-06-02 04:43:12.178856829 +0000 UTC m=+74.602627082" lastFinishedPulling="2026-06-02 04:43:14.643687069 +0000 UTC m=+77.067457309" observedRunningTime="2026-06-02 04:43:15.588218316 +0000 UTC m=+78.011988580" watchObservedRunningTime="2026-06-02 04:43:15.588574211 +0000 UTC m=+78.012344473" Jun 02 04:43:15.606248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.606195 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-76d68c6d7b-f6sxk" podStartSLOduration=1.606176187 podStartE2EDuration="1.606176187s" podCreationTimestamp="2026-06-02 04:43:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:43:15.605169369 +0000 UTC m=+78.028939631" watchObservedRunningTime="2026-06-02 04:43:15.606176187 +0000 UTC m=+78.029946451" Jun 02 04:43:15.652059 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.651986 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.280201184 podStartE2EDuration="8.651965578s" podCreationTimestamp="2026-06-02 04:43:07 +0000 UTC" firstStartedPulling="2026-06-02 04:43:09.270424409 +0000 UTC m=+71.694194663" lastFinishedPulling="2026-06-02 04:43:14.642188813 +0000 UTC m=+77.065959057" observedRunningTime="2026-06-02 04:43:15.651407834 +0000 UTC m=+78.075178099" watchObservedRunningTime="2026-06-02 04:43:15.651965578 +0000 UTC m=+78.075735842" Jun 02 04:43:15.670579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:15.670511 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-t6785" podStartSLOduration=2.8310228840000002 podStartE2EDuration="4.670491597s" podCreationTimestamp="2026-06-02 04:43:11 +0000 UTC" firstStartedPulling="2026-06-02 04:43:13.233275947 +0000 UTC m=+75.657046205" lastFinishedPulling="2026-06-02 04:43:15.072744665 +0000 UTC m=+77.496514918" observedRunningTime="2026-06-02 04:43:15.668569734 +0000 UTC m=+78.092339995" watchObservedRunningTime="2026-06-02 04:43:15.670491597 +0000 UTC m=+78.094261861" Jun 02 04:43:18.481216 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:18.481186 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66dc564ff9-6fcj8" Jun 02 04:43:22.596165 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.594578 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76d68c6d7b-f6sxk"] Jun 02 04:43:22.628135 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.628101 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-656b6468cb-8lg24"] Jun 02 04:43:22.631048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.631029 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.644865 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.644838 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656b6468cb-8lg24"] Jun 02 04:43:22.788992 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.788947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-console-config\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.789184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.789026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-oauth-serving-cert\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.789184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.789052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-oauth-config\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.789184 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.789090 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-service-ca\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.789327 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.789196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-serving-cert\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.789327 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.789227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-trusted-ca-bundle\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.789327 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.789245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl5rn\" (UniqueName: \"kubernetes.io/projected/20d839af-3626-479b-99c3-b1c1543a0416-kube-api-access-rl5rn\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.889870 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.889779 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-trusted-ca-bundle\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.889870 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.889820 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rl5rn\" (UniqueName: \"kubernetes.io/projected/20d839af-3626-479b-99c3-b1c1543a0416-kube-api-access-rl5rn\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890078 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.889942 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-console-config\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890078 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890009 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-oauth-serving-cert\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890078 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890032 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-oauth-config\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890078 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890077 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-service-ca\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890316 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890105 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-serving-cert\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890696 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890670 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-console-config\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890847 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-trusted-ca-bundle\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890926 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-service-ca\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.890926 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.890829 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-oauth-serving-cert\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.892572 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.892545 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-serving-cert\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.892665 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.892571 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-oauth-config\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.899677 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.899658 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl5rn\" (UniqueName: \"kubernetes.io/projected/20d839af-3626-479b-99c3-b1c1543a0416-kube-api-access-rl5rn\") pod \"console-656b6468cb-8lg24\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:22.939726 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:22.939685 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:23.073956 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:23.073925 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656b6468cb-8lg24"] Jun 02 04:43:23.076445 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:43:23.076415 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d839af_3626_479b_99c3_b1c1543a0416.slice/crio-493af9967986f112a7762cc2e29fa036142824149b75fdf1f7469549ece2ee00 WatchSource:0}: Error finding container 493af9967986f112a7762cc2e29fa036142824149b75fdf1f7469549ece2ee00: Status 404 returned error can't find the container with id 493af9967986f112a7762cc2e29fa036142824149b75fdf1f7469549ece2ee00 Jun 02 04:43:23.599402 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:23.599368 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b6468cb-8lg24" event={"ID":"20d839af-3626-479b-99c3-b1c1543a0416","Type":"ContainerStarted","Data":"6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b"} Jun 02 04:43:23.599402 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:23.599406 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b6468cb-8lg24" event={"ID":"20d839af-3626-479b-99c3-b1c1543a0416","Type":"ContainerStarted","Data":"493af9967986f112a7762cc2e29fa036142824149b75fdf1f7469549ece2ee00"} Jun 02 04:43:23.617845 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:23.617781 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-656b6468cb-8lg24" podStartSLOduration=1.617766394 podStartE2EDuration="1.617766394s" podCreationTimestamp="2026-06-02 04:43:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:43:23.616837324 +0000 UTC m=+86.040607587" watchObservedRunningTime="2026-06-02 04:43:23.617766394 +0000 UTC m=+86.041536656" Jun 02 04:43:25.307837 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:25.307801 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:32.940519 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:32.940460 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:32.940519 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:32.940528 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:32.945452 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:32.945428 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:33.635882 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:33.635854 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:43:33.688963 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:33.688930 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c5c7db546-rkz7v"] Jun 02 04:43:40.539403 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:40.539371 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qpm4h" Jun 02 04:43:47.616489 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.616422 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-76d68c6d7b-f6sxk" podUID="b3312699-7284-4457-8d5f-7e6ff0e6732a" containerName="console" containerID="cri-o://0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb" gracePeriod=15 Jun 02 04:43:47.857464 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.857438 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76d68c6d7b-f6sxk_b3312699-7284-4457-8d5f-7e6ff0e6732a/console/0.log" Jun 02 04:43:47.857607 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.857525 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:47.907984 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.907888 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44rq2\" (UniqueName: \"kubernetes.io/projected/b3312699-7284-4457-8d5f-7e6ff0e6732a-kube-api-access-44rq2\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.907984 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.907954 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-config\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.908248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.907989 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-oauth-serving-cert\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.908248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908012 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-service-ca\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.908248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908029 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-oauth-config\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.908248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908054 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-trusted-ca-bundle\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.908248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908081 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-serving-cert\") pod \"b3312699-7284-4457-8d5f-7e6ff0e6732a\" (UID: \"b3312699-7284-4457-8d5f-7e6ff0e6732a\") " Jun 02 04:43:47.908534 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908493 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-config" (OuterVolumeSpecName: "console-config") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:47.908610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908525 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:47.908610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908500 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:47.908610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.908507 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-service-ca" (OuterVolumeSpecName: "service-ca") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:47.910479 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.910448 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:43:47.910579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.910464 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:43:47.910579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:47.910498 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3312699-7284-4457-8d5f-7e6ff0e6732a-kube-api-access-44rq2" (OuterVolumeSpecName: "kube-api-access-44rq2") pod "b3312699-7284-4457-8d5f-7e6ff0e6732a" (UID: "b3312699-7284-4457-8d5f-7e6ff0e6732a"). InnerVolumeSpecName "kube-api-access-44rq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:43:48.008981 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.008937 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44rq2\" (UniqueName: \"kubernetes.io/projected/b3312699-7284-4457-8d5f-7e6ff0e6732a-kube-api-access-44rq2\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.008981 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.008973 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.008981 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.008984 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-oauth-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.008981 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.008992 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.009279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.009002 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-oauth-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.009279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.009012 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3312699-7284-4457-8d5f-7e6ff0e6732a-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.009279 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.009021 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3312699-7284-4457-8d5f-7e6ff0e6732a-console-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:48.677181 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.677131 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76d68c6d7b-f6sxk_b3312699-7284-4457-8d5f-7e6ff0e6732a/console/0.log" Jun 02 04:43:48.677737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.677194 2566 generic.go:358] "Generic (PLEG): container finished" podID="b3312699-7284-4457-8d5f-7e6ff0e6732a" containerID="0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb" exitCode=2 Jun 02 04:43:48.677737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.677228 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d68c6d7b-f6sxk" event={"ID":"b3312699-7284-4457-8d5f-7e6ff0e6732a","Type":"ContainerDied","Data":"0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb"} Jun 02 04:43:48.677737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.677273 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d68c6d7b-f6sxk" event={"ID":"b3312699-7284-4457-8d5f-7e6ff0e6732a","Type":"ContainerDied","Data":"9c5c3ff9cdae783e491de41ce5d2d59ee5d41b7fb067b03e986880febb65c8f4"} Jun 02 04:43:48.677737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.677279 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d68c6d7b-f6sxk" Jun 02 04:43:48.677737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.677307 2566 scope.go:117] "RemoveContainer" containerID="0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb" Jun 02 04:43:48.685166 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.685150 2566 scope.go:117] "RemoveContainer" containerID="0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb" Jun 02 04:43:48.685414 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:48.685396 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb\": container with ID starting with 0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb not found: ID does not exist" containerID="0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb" Jun 02 04:43:48.685483 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.685425 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb"} err="failed to get container status \"0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb\": rpc error: code = NotFound desc = could not find container \"0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb\": container with ID starting with 0a343e7cce612f93334ddfaa711ee59aa568307c800170727eac3dcfd9807fdb not found: ID does not exist" Jun 02 04:43:48.697129 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.697104 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76d68c6d7b-f6sxk"] Jun 02 04:43:48.703528 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:48.703503 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76d68c6d7b-f6sxk"] Jun 02 04:43:50.118776 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:50.118732 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3312699-7284-4457-8d5f-7e6ff0e6732a" path="/var/lib/kubelet/pods/b3312699-7284-4457-8d5f-7e6ff0e6732a/volumes" Jun 02 04:43:58.708808 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.708768 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6c5c7db546-rkz7v" podUID="8ef7102c-dd31-41df-8aee-27f0b02d6fb2" containerName="console" containerID="cri-o://0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826" gracePeriod=15 Jun 02 04:43:58.956992 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.956971 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c5c7db546-rkz7v_8ef7102c-dd31-41df-8aee-27f0b02d6fb2/console/0.log" Jun 02 04:43:58.957113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.957031 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:58.996094 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996004 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-oauth-serving-cert\") pod \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " Jun 02 04:43:58.996094 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996046 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-serving-cert\") pod \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " Jun 02 04:43:58.996308 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996094 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2cwk\" (UniqueName: \"kubernetes.io/projected/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-kube-api-access-m2cwk\") pod \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " Jun 02 04:43:58.996308 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996122 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-service-ca\") pod \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " Jun 02 04:43:58.996308 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996189 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-oauth-config\") pod \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " Jun 02 04:43:58.996308 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996229 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-config\") pod \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\" (UID: \"8ef7102c-dd31-41df-8aee-27f0b02d6fb2\") " Jun 02 04:43:58.996605 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996574 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8ef7102c-dd31-41df-8aee-27f0b02d6fb2" (UID: "8ef7102c-dd31-41df-8aee-27f0b02d6fb2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:58.996724 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996654 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-service-ca" (OuterVolumeSpecName: "service-ca") pod "8ef7102c-dd31-41df-8aee-27f0b02d6fb2" (UID: "8ef7102c-dd31-41df-8aee-27f0b02d6fb2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:58.996792 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.996768 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-config" (OuterVolumeSpecName: "console-config") pod "8ef7102c-dd31-41df-8aee-27f0b02d6fb2" (UID: "8ef7102c-dd31-41df-8aee-27f0b02d6fb2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:43:58.998357 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.998328 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-kube-api-access-m2cwk" (OuterVolumeSpecName: "kube-api-access-m2cwk") pod "8ef7102c-dd31-41df-8aee-27f0b02d6fb2" (UID: "8ef7102c-dd31-41df-8aee-27f0b02d6fb2"). InnerVolumeSpecName "kube-api-access-m2cwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:43:58.998357 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.998338 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8ef7102c-dd31-41df-8aee-27f0b02d6fb2" (UID: "8ef7102c-dd31-41df-8aee-27f0b02d6fb2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:43:58.998495 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:58.998359 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8ef7102c-dd31-41df-8aee-27f0b02d6fb2" (UID: "8ef7102c-dd31-41df-8aee-27f0b02d6fb2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:43:59.097221 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.097179 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-oauth-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:59.097221 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.097216 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:59.097221 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.097229 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2cwk\" (UniqueName: \"kubernetes.io/projected/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-kube-api-access-m2cwk\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:59.097465 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.097242 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:59.097465 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.097258 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-oauth-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:59.097465 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.097270 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ef7102c-dd31-41df-8aee-27f0b02d6fb2-console-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:43:59.709286 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.709257 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c5c7db546-rkz7v_8ef7102c-dd31-41df-8aee-27f0b02d6fb2/console/0.log" Jun 02 04:43:59.709673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.709294 2566 generic.go:358] "Generic (PLEG): container finished" podID="8ef7102c-dd31-41df-8aee-27f0b02d6fb2" containerID="0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826" exitCode=2 Jun 02 04:43:59.709673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.709323 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5c7db546-rkz7v" event={"ID":"8ef7102c-dd31-41df-8aee-27f0b02d6fb2","Type":"ContainerDied","Data":"0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826"} Jun 02 04:43:59.709673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.709364 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5c7db546-rkz7v" event={"ID":"8ef7102c-dd31-41df-8aee-27f0b02d6fb2","Type":"ContainerDied","Data":"7516eea6c16569e645df36e8fa937de7b6bdc85b97cf2a510d726249922e442c"} Jun 02 04:43:59.709673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.709364 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5c7db546-rkz7v" Jun 02 04:43:59.709673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.709376 2566 scope.go:117] "RemoveContainer" containerID="0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826" Jun 02 04:43:59.717341 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.717326 2566 scope.go:117] "RemoveContainer" containerID="0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826" Jun 02 04:43:59.717581 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:43:59.717563 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826\": container with ID starting with 0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826 not found: ID does not exist" containerID="0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826" Jun 02 04:43:59.717624 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.717591 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826"} err="failed to get container status \"0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826\": rpc error: code = NotFound desc = could not find container \"0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826\": container with ID starting with 0f7b188c06cfd64c30757cd8600bab33a267c052ba2d111c6c96123e1b37e826 not found: ID does not exist" Jun 02 04:43:59.730206 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.730184 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c5c7db546-rkz7v"] Jun 02 04:43:59.735768 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:43:59.735745 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c5c7db546-rkz7v"] Jun 02 04:44:00.119022 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:00.118988 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ef7102c-dd31-41df-8aee-27f0b02d6fb2" path="/var/lib/kubelet/pods/8ef7102c-dd31-41df-8aee-27f0b02d6fb2/volumes" Jun 02 04:44:26.848264 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848229 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:44:26.848940 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848898 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="alertmanager" containerID="cri-o://88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e" gracePeriod=120 Jun 02 04:44:26.849083 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848968 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy" containerID="cri-o://45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3" gracePeriod=120 Jun 02 04:44:26.849083 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848995 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="prom-label-proxy" containerID="cri-o://a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01" gracePeriod=120 Jun 02 04:44:26.849083 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848995 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="config-reloader" containerID="cri-o://d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4" gracePeriod=120 Jun 02 04:44:26.849083 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848955 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-web" containerID="cri-o://2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2" gracePeriod=120 Jun 02 04:44:26.849332 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:26.848923 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-metric" containerID="cri-o://0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d" gracePeriod=120 Jun 02 04:44:27.793669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793632 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01" exitCode=0 Jun 02 04:44:27.793669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793662 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3" exitCode=0 Jun 02 04:44:27.793669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793669 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4" exitCode=0 Jun 02 04:44:27.793669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793675 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e" exitCode=0 Jun 02 04:44:27.793967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793703 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01"} Jun 02 04:44:27.793967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793745 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3"} Jun 02 04:44:27.793967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793761 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4"} Jun 02 04:44:27.793967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:27.793774 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e"} Jun 02 04:44:28.095187 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.095164 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.242723 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242667 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-trusted-ca-bundle\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242723 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242721 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45thl\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-kube-api-access-45thl\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242752 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-volume\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242774 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-cluster-tls-config\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242799 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-web\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242854 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-main-db\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242882 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242909 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-metrics-client-ca\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.242966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.242961 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-web-config\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.243356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243001 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.243356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243039 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.243356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243066 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-out\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.243356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243090 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-tls-assets\") pod \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\" (UID: \"748ef5cb-4cfe-485d-9090-3a5bdc407c5e\") " Jun 02 04:44:28.243356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243165 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:44:28.243595 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243357 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:44:28.243595 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.243412 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.244488 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.244462 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:44:28.245631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.245601 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-volume" (OuterVolumeSpecName: "config-volume") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.246097 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.246056 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-kube-api-access-45thl" (OuterVolumeSpecName: "kube-api-access-45thl") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "kube-api-access-45thl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:44:28.246401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.246372 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.246493 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.246438 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:44:28.246651 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.246629 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.246832 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.246801 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.247326 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.247295 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.247420 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.247397 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-out" (OuterVolumeSpecName: "config-out") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:44:28.249864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.249835 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.256439 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.256415 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-web-config" (OuterVolumeSpecName: "web-config") pod "748ef5cb-4cfe-485d-9090-3a5bdc407c5e" (UID: "748ef5cb-4cfe-485d-9090-3a5bdc407c5e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344388 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344421 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-main-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344435 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-out\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344445 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-tls-assets\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344453 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45thl\" (UniqueName: \"kubernetes.io/projected/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-kube-api-access-45thl\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344465 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-config-volume\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344474 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-cluster-tls-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344484 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344493 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-alertmanager-main-db\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344505 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344514 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-metrics-client-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.344775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.344523 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/748ef5cb-4cfe-485d-9090-3a5bdc407c5e-web-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:44:28.799724 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799644 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d" exitCode=0 Jun 02 04:44:28.799724 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799667 2566 generic.go:358] "Generic (PLEG): container finished" podID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerID="2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2" exitCode=0 Jun 02 04:44:28.799898 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799720 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d"} Jun 02 04:44:28.799898 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799771 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2"} Jun 02 04:44:28.799898 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799787 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"748ef5cb-4cfe-485d-9090-3a5bdc407c5e","Type":"ContainerDied","Data":"605acab68b01e589208f4fc8ea5f5268885ad3fd40e9a177c909923aca60e464"} Jun 02 04:44:28.799898 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799804 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.799898 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.799807 2566 scope.go:117] "RemoveContainer" containerID="a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01" Jun 02 04:44:28.806971 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.806946 2566 scope.go:117] "RemoveContainer" containerID="0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d" Jun 02 04:44:28.813497 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.813475 2566 scope.go:117] "RemoveContainer" containerID="45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3" Jun 02 04:44:28.819683 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.819665 2566 scope.go:117] "RemoveContainer" containerID="2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2" Jun 02 04:44:28.822435 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.822411 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:44:28.826635 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.826622 2566 scope.go:117] "RemoveContainer" containerID="d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4" Jun 02 04:44:28.829154 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.829117 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:44:28.834375 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.834352 2566 scope.go:117] "RemoveContainer" containerID="88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e" Jun 02 04:44:28.840898 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.840880 2566 scope.go:117] "RemoveContainer" containerID="1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626" Jun 02 04:44:28.847217 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.847203 2566 scope.go:117] "RemoveContainer" containerID="a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01" Jun 02 04:44:28.847579 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.847552 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01\": container with ID starting with a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01 not found: ID does not exist" containerID="a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01" Jun 02 04:44:28.847649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.847592 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01"} err="failed to get container status \"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01\": rpc error: code = NotFound desc = could not find container \"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01\": container with ID starting with a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01 not found: ID does not exist" Jun 02 04:44:28.847649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.847625 2566 scope.go:117] "RemoveContainer" containerID="0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d" Jun 02 04:44:28.847924 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.847895 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d\": container with ID starting with 0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d not found: ID does not exist" containerID="0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d" Jun 02 04:44:28.848018 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.847921 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d"} err="failed to get container status \"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d\": rpc error: code = NotFound desc = could not find container \"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d\": container with ID starting with 0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d not found: ID does not exist" Jun 02 04:44:28.848018 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.847952 2566 scope.go:117] "RemoveContainer" containerID="45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3" Jun 02 04:44:28.848235 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.848216 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3\": container with ID starting with 45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3 not found: ID does not exist" containerID="45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3" Jun 02 04:44:28.848286 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.848243 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3"} err="failed to get container status \"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3\": rpc error: code = NotFound desc = could not find container \"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3\": container with ID starting with 45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3 not found: ID does not exist" Jun 02 04:44:28.848286 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.848261 2566 scope.go:117] "RemoveContainer" containerID="2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2" Jun 02 04:44:28.848514 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.848495 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2\": container with ID starting with 2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2 not found: ID does not exist" containerID="2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2" Jun 02 04:44:28.848568 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.848518 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2"} err="failed to get container status \"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2\": rpc error: code = NotFound desc = could not find container \"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2\": container with ID starting with 2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2 not found: ID does not exist" Jun 02 04:44:28.848568 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.848533 2566 scope.go:117] "RemoveContainer" containerID="d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4" Jun 02 04:44:28.848783 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.848759 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4\": container with ID starting with d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4 not found: ID does not exist" containerID="d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4" Jun 02 04:44:28.848837 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.848791 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4"} err="failed to get container status \"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4\": rpc error: code = NotFound desc = could not find container \"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4\": container with ID starting with d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4 not found: ID does not exist" Jun 02 04:44:28.848837 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.848805 2566 scope.go:117] "RemoveContainer" containerID="88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e" Jun 02 04:44:28.849012 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.848995 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e\": container with ID starting with 88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e not found: ID does not exist" containerID="88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e" Jun 02 04:44:28.849048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849016 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e"} err="failed to get container status \"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e\": rpc error: code = NotFound desc = could not find container \"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e\": container with ID starting with 88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e not found: ID does not exist" Jun 02 04:44:28.849048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849029 2566 scope.go:117] "RemoveContainer" containerID="1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626" Jun 02 04:44:28.849311 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:44:28.849293 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626\": container with ID starting with 1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626 not found: ID does not exist" containerID="1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626" Jun 02 04:44:28.849372 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849315 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626"} err="failed to get container status \"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626\": rpc error: code = NotFound desc = could not find container \"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626\": container with ID starting with 1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626 not found: ID does not exist" Jun 02 04:44:28.849372 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849336 2566 scope.go:117] "RemoveContainer" containerID="a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01" Jun 02 04:44:28.849566 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849547 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01"} err="failed to get container status \"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01\": rpc error: code = NotFound desc = could not find container \"a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01\": container with ID starting with a9e155ac436e6cfc98017ba5162f2b384aebb34a48b4a97ee7ec0125a4968f01 not found: ID does not exist" Jun 02 04:44:28.849612 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849567 2566 scope.go:117] "RemoveContainer" containerID="0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d" Jun 02 04:44:28.849772 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849755 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d"} err="failed to get container status \"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d\": rpc error: code = NotFound desc = could not find container \"0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d\": container with ID starting with 0c267f10e18d4daf29224cd1282267f02f35357838860743dc2082e2b126032d not found: ID does not exist" Jun 02 04:44:28.849816 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849772 2566 scope.go:117] "RemoveContainer" containerID="45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3" Jun 02 04:44:28.849959 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849944 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3"} err="failed to get container status \"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3\": rpc error: code = NotFound desc = could not find container \"45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3\": container with ID starting with 45d018a894ca9777f373959ba6a67b85f82e655f86c9e317138bdeea8706b7d3 not found: ID does not exist" Jun 02 04:44:28.850000 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.849959 2566 scope.go:117] "RemoveContainer" containerID="2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2" Jun 02 04:44:28.850173 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850134 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2"} err="failed to get container status \"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2\": rpc error: code = NotFound desc = could not find container \"2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2\": container with ID starting with 2748ef6b280525af77550a8a3d1833fd8b3ee07a011296b9cae826413cac0cb2 not found: ID does not exist" Jun 02 04:44:28.850247 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850177 2566 scope.go:117] "RemoveContainer" containerID="d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4" Jun 02 04:44:28.850385 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850368 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4"} err="failed to get container status \"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4\": rpc error: code = NotFound desc = could not find container \"d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4\": container with ID starting with d7fcb0372e99840a28ac79c5cbda35a99f19a82f2de5f12b0f640603286ca7f4 not found: ID does not exist" Jun 02 04:44:28.850447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850386 2566 scope.go:117] "RemoveContainer" containerID="88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e" Jun 02 04:44:28.850578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850562 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e"} err="failed to get container status \"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e\": rpc error: code = NotFound desc = could not find container \"88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e\": container with ID starting with 88df2efa16015ffc2ae988dfabff293cc0a4ba19eec78aa1ab35ccbd0799845e not found: ID does not exist" Jun 02 04:44:28.850578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850577 2566 scope.go:117] "RemoveContainer" containerID="1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626" Jun 02 04:44:28.850734 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.850717 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626"} err="failed to get container status \"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626\": rpc error: code = NotFound desc = could not find container \"1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626\": container with ID starting with 1e5efa4c2331ba4482434405b63295c092b2ec13313a0a20feb7b8dae70c2626 not found: ID does not exist" Jun 02 04:44:28.857536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857513 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:44:28.857889 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857874 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="alertmanager" Jun 02 04:44:28.857945 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857893 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="alertmanager" Jun 02 04:44:28.857945 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857909 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy" Jun 02 04:44:28.857945 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857917 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy" Jun 02 04:44:28.857945 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857931 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="config-reloader" Jun 02 04:44:28.857945 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857940 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="config-reloader" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857950 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-web" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857962 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-web" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857979 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-metric" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.857988 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-metric" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858004 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef7102c-dd31-41df-8aee-27f0b02d6fb2" containerName="console" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858012 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef7102c-dd31-41df-8aee-27f0b02d6fb2" containerName="console" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858022 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3312699-7284-4457-8d5f-7e6ff0e6732a" containerName="console" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858030 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3312699-7284-4457-8d5f-7e6ff0e6732a" containerName="console" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858042 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="init-config-reloader" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858050 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="init-config-reloader" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858064 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="prom-label-proxy" Jun 02 04:44:28.858100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858073 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="prom-label-proxy" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858132 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-web" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858160 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="b3312699-7284-4457-8d5f-7e6ff0e6732a" containerName="console" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858171 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="prom-label-proxy" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858180 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="8ef7102c-dd31-41df-8aee-27f0b02d6fb2" containerName="console" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858189 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy-metric" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858200 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="config-reloader" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858208 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="kube-rbac-proxy" Jun 02 04:44:28.858474 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.858218 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" containerName="alertmanager" Jun 02 04:44:28.863310 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.863293 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.865843 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.865786 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6db8r\"" Jun 02 04:44:28.865977 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.865964 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 04:44:28.866044 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.865997 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 04:44:28.866398 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.866381 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 04:44:28.866521 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.866504 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 04:44:28.866589 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.866571 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 04:44:28.866762 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.866748 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 04:44:28.866828 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.866809 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 04:44:28.866889 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.866867 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 04:44:28.872589 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.872563 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 04:44:28.873665 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.873644 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:44:28.948758 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948714 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.948758 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lh7t\" (UniqueName: \"kubernetes.io/projected/4ce50496-003d-4540-9dc3-f4182a66fef9-kube-api-access-9lh7t\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948865 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ce50496-003d-4540-9dc3-f4182a66fef9-config-out\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ce50496-003d-4540-9dc3-f4182a66fef9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ce50496-003d-4540-9dc3-f4182a66fef9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.948985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.949067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-web-config\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.949102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.949131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ce50496-003d-4540-9dc3-f4182a66fef9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:28.949272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:28.949182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ce50496-003d-4540-9dc3-f4182a66fef9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.049930 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.049838 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9lh7t\" (UniqueName: \"kubernetes.io/projected/4ce50496-003d-4540-9dc3-f4182a66fef9-kube-api-access-9lh7t\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.049930 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.049885 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.049930 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.049905 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ce50496-003d-4540-9dc3-f4182a66fef9-config-out\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050193 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050027 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ce50496-003d-4540-9dc3-f4182a66fef9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050193 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050077 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ce50496-003d-4540-9dc3-f4182a66fef9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050193 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050109 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050193 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050171 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050232 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-web-config\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050264 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050292 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ce50496-003d-4540-9dc3-f4182a66fef9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050323 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ce50496-003d-4540-9dc3-f4182a66fef9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050349 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.050401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050378 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.052121 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.050833 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ce50496-003d-4540-9dc3-f4182a66fef9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.052121 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.051768 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ce50496-003d-4540-9dc3-f4182a66fef9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.052807 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.052721 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ce50496-003d-4540-9dc3-f4182a66fef9-config-out\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.052913 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.052851 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ce50496-003d-4540-9dc3-f4182a66fef9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.053167 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.053081 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.053261 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.053208 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.053261 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.053238 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ce50496-003d-4540-9dc3-f4182a66fef9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.053491 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.053465 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.053575 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.053556 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-web-config\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.053618 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.053565 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.054363 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.054344 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.055106 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.055088 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ce50496-003d-4540-9dc3-f4182a66fef9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.058936 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.058917 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lh7t\" (UniqueName: \"kubernetes.io/projected/4ce50496-003d-4540-9dc3-f4182a66fef9-kube-api-access-9lh7t\") pod \"alertmanager-main-0\" (UID: \"4ce50496-003d-4540-9dc3-f4182a66fef9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.173078 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.173039 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 04:44:29.307918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.307834 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 04:44:29.311545 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:44:29.311511 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce50496_003d_4540_9dc3_f4182a66fef9.slice/crio-36c18d8236337890b2a9bfea1d3146a981d8089f158aa82c8093cec8cdc208a5 WatchSource:0}: Error finding container 36c18d8236337890b2a9bfea1d3146a981d8089f158aa82c8093cec8cdc208a5: Status 404 returned error can't find the container with id 36c18d8236337890b2a9bfea1d3146a981d8089f158aa82c8093cec8cdc208a5 Jun 02 04:44:29.809968 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.809925 2566 generic.go:358] "Generic (PLEG): container finished" podID="4ce50496-003d-4540-9dc3-f4182a66fef9" containerID="36281a648bcc1feaf7ed936a58647b0bf9263574b16a74a2e36b270b1678c7b6" exitCode=0 Jun 02 04:44:29.810164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.810009 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerDied","Data":"36281a648bcc1feaf7ed936a58647b0bf9263574b16a74a2e36b270b1678c7b6"} Jun 02 04:44:29.810164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:29.810045 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"36c18d8236337890b2a9bfea1d3146a981d8089f158aa82c8093cec8cdc208a5"} Jun 02 04:44:30.119117 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.119088 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="748ef5cb-4cfe-485d-9090-3a5bdc407c5e" path="/var/lib/kubelet/pods/748ef5cb-4cfe-485d-9090-3a5bdc407c5e/volumes" Jun 02 04:44:30.818882 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.818849 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"489d373d9f5643c0ac8fb40a6a07b41dfed87d8414211cf3e5d686196da7e371"} Jun 02 04:44:30.818882 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.818884 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"13acba29705ec8409fe46b0cf11ba00d767efc02188206fe786c18c23ce7260d"} Jun 02 04:44:30.819301 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.818895 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"43951caacd967c007a4029415714d5d82b5ae04263e81f9262feb8e8b6561a5b"} Jun 02 04:44:30.819301 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.818905 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"07d273f0991050f54208bea9e1c4a258aefc1c1a2d67199125e7e5f26890712c"} Jun 02 04:44:30.819301 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.818913 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"300412f07faaf45d135d96fca28ed76f97333219a5175b327a89476e203152a2"} Jun 02 04:44:30.819301 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.818921 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ce50496-003d-4540-9dc3-f4182a66fef9","Type":"ContainerStarted","Data":"a64349c8255924b070907f2401660afe270a8d4aec8d998274522f480920c095"} Jun 02 04:44:30.851199 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:30.851126 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8511120119999998 podStartE2EDuration="2.851112012s" podCreationTimestamp="2026-06-02 04:44:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:44:30.848099768 +0000 UTC m=+153.271870030" watchObservedRunningTime="2026-06-02 04:44:30.851112012 +0000 UTC m=+153.274882274" Jun 02 04:44:51.646196 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.646095 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d6dd5966d-bsqll"] Jun 02 04:44:51.649662 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.649643 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.660493 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.660465 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d6dd5966d-bsqll"] Jun 02 04:44:51.726085 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726049 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-serving-cert\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.726251 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-service-ca\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.726251 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmgt5\" (UniqueName: \"kubernetes.io/projected/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-kube-api-access-zmgt5\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.726251 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-trusted-ca-bundle\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.726370 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-oauth-config\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.726370 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726283 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-oauth-serving-cert\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.726370 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.726299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-config\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.826914 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.826875 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-service-ca\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.826914 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.826913 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zmgt5\" (UniqueName: \"kubernetes.io/projected/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-kube-api-access-zmgt5\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827117 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.826950 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-trusted-ca-bundle\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827117 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.826976 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-oauth-config\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827117 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827002 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-oauth-serving-cert\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827117 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827029 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-config\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827117 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827071 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-serving-cert\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827694 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827663 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-service-ca\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827817 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827798 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-trusted-ca-bundle\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827926 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827899 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-oauth-serving-cert\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.827978 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.827933 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-config\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.829440 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.829416 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-oauth-config\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.829538 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.829522 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-serving-cert\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.835024 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.835000 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmgt5\" (UniqueName: \"kubernetes.io/projected/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-kube-api-access-zmgt5\") pod \"console-6d6dd5966d-bsqll\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:51.958699 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:51.958601 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:44:52.085970 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:52.085929 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d6dd5966d-bsqll"] Jun 02 04:44:52.092249 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:44:52.092217 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2ac6b4_6f68_40d4_a526_89de8e2ee227.slice/crio-db9f189b211f19d5a5e613399780165940d5103f93338be927d4e5c591d76829 WatchSource:0}: Error finding container db9f189b211f19d5a5e613399780165940d5103f93338be927d4e5c591d76829: Status 404 returned error can't find the container with id db9f189b211f19d5a5e613399780165940d5103f93338be927d4e5c591d76829 Jun 02 04:44:52.880768 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:52.880729 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dd5966d-bsqll" event={"ID":"4c2ac6b4-6f68-40d4-a526-89de8e2ee227","Type":"ContainerStarted","Data":"6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097"} Jun 02 04:44:52.880768 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:52.880769 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dd5966d-bsqll" event={"ID":"4c2ac6b4-6f68-40d4-a526-89de8e2ee227","Type":"ContainerStarted","Data":"db9f189b211f19d5a5e613399780165940d5103f93338be927d4e5c591d76829"} Jun 02 04:44:52.913479 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:44:52.913304 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6d6dd5966d-bsqll" podStartSLOduration=1.913283126 podStartE2EDuration="1.913283126s" podCreationTimestamp="2026-06-02 04:44:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:44:52.911450177 +0000 UTC m=+175.335220439" watchObservedRunningTime="2026-06-02 04:44:52.913283126 +0000 UTC m=+175.337053391" Jun 02 04:45:01.959594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:01.959529 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:45:01.959594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:01.959600 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:45:01.964560 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:01.964532 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:45:02.912948 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:02.912912 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:45:02.965310 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:02.965279 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-656b6468cb-8lg24"] Jun 02 04:45:21.380690 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.380654 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wc5md"] Jun 02 04:45:21.384130 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.384051 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.386399 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.386376 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 04:45:21.392779 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.392753 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wc5md"] Jun 02 04:45:21.461988 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.461947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad29c16f-6b87-412b-b764-b39c6f516012-dbus\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.461988 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.461995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad29c16f-6b87-412b-b764-b39c6f516012-kubelet-config\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.462248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.462073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad29c16f-6b87-412b-b764-b39c6f516012-original-pull-secret\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.563463 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.563419 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad29c16f-6b87-412b-b764-b39c6f516012-dbus\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.563639 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.563477 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad29c16f-6b87-412b-b764-b39c6f516012-kubelet-config\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.563639 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.563530 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad29c16f-6b87-412b-b764-b39c6f516012-original-pull-secret\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.563639 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.563613 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad29c16f-6b87-412b-b764-b39c6f516012-kubelet-config\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.563786 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.563614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad29c16f-6b87-412b-b764-b39c6f516012-dbus\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.565879 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.565858 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad29c16f-6b87-412b-b764-b39c6f516012-original-pull-secret\") pod \"global-pull-secret-syncer-wc5md\" (UID: \"ad29c16f-6b87-412b-b764-b39c6f516012\") " pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.693281 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.693186 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wc5md" Jun 02 04:45:21.817700 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.817666 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wc5md"] Jun 02 04:45:21.820685 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:45:21.820657 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad29c16f_6b87_412b_b764_b39c6f516012.slice/crio-b877f7e668b0309113d6476986caa21f8b660e40695a59ba76de9452f5c5562c WatchSource:0}: Error finding container b877f7e668b0309113d6476986caa21f8b660e40695a59ba76de9452f5c5562c: Status 404 returned error can't find the container with id b877f7e668b0309113d6476986caa21f8b660e40695a59ba76de9452f5c5562c Jun 02 04:45:21.961401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:21.961309 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wc5md" event={"ID":"ad29c16f-6b87-412b-b764-b39c6f516012","Type":"ContainerStarted","Data":"b877f7e668b0309113d6476986caa21f8b660e40695a59ba76de9452f5c5562c"} Jun 02 04:45:25.975580 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:25.975534 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wc5md" event={"ID":"ad29c16f-6b87-412b-b764-b39c6f516012","Type":"ContainerStarted","Data":"b3f19d30af25cca97afaf62c52a24f74b1323c4b470b4625084cd96e5bdbb12d"} Jun 02 04:45:25.993962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:25.993910 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wc5md" podStartSLOduration=1.051698159 podStartE2EDuration="4.993896063s" podCreationTimestamp="2026-06-02 04:45:21 +0000 UTC" firstStartedPulling="2026-06-02 04:45:21.822169195 +0000 UTC m=+204.245939440" lastFinishedPulling="2026-06-02 04:45:25.764367102 +0000 UTC m=+208.188137344" observedRunningTime="2026-06-02 04:45:25.99349935 +0000 UTC m=+208.417269612" watchObservedRunningTime="2026-06-02 04:45:25.993896063 +0000 UTC m=+208.417666350" Jun 02 04:45:27.985230 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:27.985193 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-656b6468cb-8lg24" podUID="20d839af-3626-479b-99c3-b1c1543a0416" containerName="console" containerID="cri-o://6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b" gracePeriod=15 Jun 02 04:45:28.224676 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.224651 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-656b6468cb-8lg24_20d839af-3626-479b-99c3-b1c1543a0416/console/0.log" Jun 02 04:45:28.224817 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.224722 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:45:28.323040 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323004 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-serving-cert\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323242 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323064 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-console-config\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323242 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323089 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-oauth-serving-cert\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323242 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323112 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-oauth-config\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323242 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323189 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-service-ca\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323445 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323247 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-trusted-ca-bundle\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323445 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323279 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rl5rn\" (UniqueName: \"kubernetes.io/projected/20d839af-3626-479b-99c3-b1c1543a0416-kube-api-access-rl5rn\") pod \"20d839af-3626-479b-99c3-b1c1543a0416\" (UID: \"20d839af-3626-479b-99c3-b1c1543a0416\") " Jun 02 04:45:28.323701 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323646 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-console-config" (OuterVolumeSpecName: "console-config") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:45:28.323701 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323677 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:45:28.323701 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323649 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:45:28.323879 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.323695 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-service-ca" (OuterVolumeSpecName: "service-ca") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:45:28.325505 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.325481 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:45:28.325594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.325483 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d839af-3626-479b-99c3-b1c1543a0416-kube-api-access-rl5rn" (OuterVolumeSpecName: "kube-api-access-rl5rn") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "kube-api-access-rl5rn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:45:28.325594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.325549 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20d839af-3626-479b-99c3-b1c1543a0416" (UID: "20d839af-3626-479b-99c3-b1c1543a0416"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:45:28.424673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424633 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.424673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424666 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.424673 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424677 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl5rn\" (UniqueName: \"kubernetes.io/projected/20d839af-3626-479b-99c3-b1c1543a0416-kube-api-access-rl5rn\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.424899 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424688 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.424899 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424697 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-console-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.424899 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424706 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20d839af-3626-479b-99c3-b1c1543a0416-oauth-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.424899 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.424715 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20d839af-3626-479b-99c3-b1c1543a0416-console-oauth-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:28.987518 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.987488 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-656b6468cb-8lg24_20d839af-3626-479b-99c3-b1c1543a0416/console/0.log" Jun 02 04:45:28.988042 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.987535 2566 generic.go:358] "Generic (PLEG): container finished" podID="20d839af-3626-479b-99c3-b1c1543a0416" containerID="6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b" exitCode=2 Jun 02 04:45:28.988042 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.987598 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b6468cb-8lg24" event={"ID":"20d839af-3626-479b-99c3-b1c1543a0416","Type":"ContainerDied","Data":"6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b"} Jun 02 04:45:28.988042 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.987637 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b6468cb-8lg24" Jun 02 04:45:28.988042 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.987651 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b6468cb-8lg24" event={"ID":"20d839af-3626-479b-99c3-b1c1543a0416","Type":"ContainerDied","Data":"493af9967986f112a7762cc2e29fa036142824149b75fdf1f7469549ece2ee00"} Jun 02 04:45:28.988042 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.987672 2566 scope.go:117] "RemoveContainer" containerID="6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b" Jun 02 04:45:28.996283 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.996262 2566 scope.go:117] "RemoveContainer" containerID="6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b" Jun 02 04:45:28.996531 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:45:28.996510 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b\": container with ID starting with 6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b not found: ID does not exist" containerID="6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b" Jun 02 04:45:28.996624 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:28.996537 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b"} err="failed to get container status \"6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b\": rpc error: code = NotFound desc = could not find container \"6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b\": container with ID starting with 6f767c8d6cc12fa33a98df9fe4bb4c1d031d2ef6a782674e4ae7e17abda5d09b not found: ID does not exist" Jun 02 04:45:29.010256 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:29.010229 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-656b6468cb-8lg24"] Jun 02 04:45:29.014732 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:29.014706 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-656b6468cb-8lg24"] Jun 02 04:45:30.118305 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:30.118266 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d839af-3626-479b-99c3-b1c1543a0416" path="/var/lib/kubelet/pods/20d839af-3626-479b-99c3-b1c1543a0416/volumes" Jun 02 04:45:34.154302 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.154269 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq"] Jun 02 04:45:34.154708 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.154528 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d839af-3626-479b-99c3-b1c1543a0416" containerName="console" Jun 02 04:45:34.154708 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.154538 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d839af-3626-479b-99c3-b1c1543a0416" containerName="console" Jun 02 04:45:34.154708 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.154591 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="20d839af-3626-479b-99c3-b1c1543a0416" containerName="console" Jun 02 04:45:34.159229 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.159209 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.161723 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.161699 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7hkf\"" Jun 02 04:45:34.161802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.161712 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 04:45:34.162489 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.162468 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 04:45:34.167248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.167221 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq"] Jun 02 04:45:34.271541 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.271495 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-447sr\" (UniqueName: \"kubernetes.io/projected/6bdf4358-0000-4257-8a83-cd7693e76fe4-kube-api-access-447sr\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.271742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.271555 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.271742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.271612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.372362 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.372325 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.372543 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.372400 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-447sr\" (UniqueName: \"kubernetes.io/projected/6bdf4358-0000-4257-8a83-cd7693e76fe4-kube-api-access-447sr\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.372543 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.372465 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.372748 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.372726 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.372844 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.372821 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.381333 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.381300 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-447sr\" (UniqueName: \"kubernetes.io/projected/6bdf4358-0000-4257-8a83-cd7693e76fe4-kube-api-access-447sr\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.470038 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.469941 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:34.594874 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:34.594816 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq"] Jun 02 04:45:34.598569 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:45:34.598538 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdf4358_0000_4257_8a83_cd7693e76fe4.slice/crio-53f7aa225097a44833451dab881a0aa0702ecf812f922daa93786cc129b15abc WatchSource:0}: Error finding container 53f7aa225097a44833451dab881a0aa0702ecf812f922daa93786cc129b15abc: Status 404 returned error can't find the container with id 53f7aa225097a44833451dab881a0aa0702ecf812f922daa93786cc129b15abc Jun 02 04:45:35.007561 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:35.007525 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" event={"ID":"6bdf4358-0000-4257-8a83-cd7693e76fe4","Type":"ContainerStarted","Data":"53f7aa225097a44833451dab881a0aa0702ecf812f922daa93786cc129b15abc"} Jun 02 04:45:42.031188 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:42.031126 2566 generic.go:358] "Generic (PLEG): container finished" podID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerID="1f4e4ee1827e2aa1d58e96dcdb9153921a44c8ecbfc540ebe2d20c812f8116ce" exitCode=0 Jun 02 04:45:42.031564 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:42.031215 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" event={"ID":"6bdf4358-0000-4257-8a83-cd7693e76fe4","Type":"ContainerDied","Data":"1f4e4ee1827e2aa1d58e96dcdb9153921a44c8ecbfc540ebe2d20c812f8116ce"} Jun 02 04:45:44.038776 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:44.038743 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" event={"ID":"6bdf4358-0000-4257-8a83-cd7693e76fe4","Type":"ContainerStarted","Data":"a86d5d8db8bfa2ac447debc95a0875b326aa345654b3caa3e2df60a97088c148"} Jun 02 04:45:45.043030 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:45.042993 2566 generic.go:358] "Generic (PLEG): container finished" podID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerID="a86d5d8db8bfa2ac447debc95a0875b326aa345654b3caa3e2df60a97088c148" exitCode=0 Jun 02 04:45:45.043456 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:45.043090 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" event={"ID":"6bdf4358-0000-4257-8a83-cd7693e76fe4","Type":"ContainerDied","Data":"a86d5d8db8bfa2ac447debc95a0875b326aa345654b3caa3e2df60a97088c148"} Jun 02 04:45:52.067545 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:52.067503 2566 generic.go:358] "Generic (PLEG): container finished" podID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerID="9cdbde1af3adac9fd1a7b3625b5520a49d9c36dd711dcb19070cf8a4da9733a2" exitCode=0 Jun 02 04:45:52.067941 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:52.067562 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" event={"ID":"6bdf4358-0000-4257-8a83-cd7693e76fe4","Type":"ContainerDied","Data":"9cdbde1af3adac9fd1a7b3625b5520a49d9c36dd711dcb19070cf8a4da9733a2"} Jun 02 04:45:53.195848 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.195820 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:53.230979 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.230933 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-bundle\") pod \"6bdf4358-0000-4257-8a83-cd7693e76fe4\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " Jun 02 04:45:53.231162 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.231074 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-447sr\" (UniqueName: \"kubernetes.io/projected/6bdf4358-0000-4257-8a83-cd7693e76fe4-kube-api-access-447sr\") pod \"6bdf4358-0000-4257-8a83-cd7693e76fe4\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " Jun 02 04:45:53.231162 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.231108 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-util\") pod \"6bdf4358-0000-4257-8a83-cd7693e76fe4\" (UID: \"6bdf4358-0000-4257-8a83-cd7693e76fe4\") " Jun 02 04:45:53.231737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.231709 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-bundle" (OuterVolumeSpecName: "bundle") pod "6bdf4358-0000-4257-8a83-cd7693e76fe4" (UID: "6bdf4358-0000-4257-8a83-cd7693e76fe4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:45:53.233403 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.233377 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bdf4358-0000-4257-8a83-cd7693e76fe4-kube-api-access-447sr" (OuterVolumeSpecName: "kube-api-access-447sr") pod "6bdf4358-0000-4257-8a83-cd7693e76fe4" (UID: "6bdf4358-0000-4257-8a83-cd7693e76fe4"). InnerVolumeSpecName "kube-api-access-447sr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:45:53.235801 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.235775 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-util" (OuterVolumeSpecName: "util") pod "6bdf4358-0000-4257-8a83-cd7693e76fe4" (UID: "6bdf4358-0000-4257-8a83-cd7693e76fe4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:45:53.331918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.331829 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-447sr\" (UniqueName: \"kubernetes.io/projected/6bdf4358-0000-4257-8a83-cd7693e76fe4-kube-api-access-447sr\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:53.331918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.331863 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:53.331918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:53.331877 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdf4358-0000-4257-8a83-cd7693e76fe4-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:45:54.074418 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:54.074384 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" event={"ID":"6bdf4358-0000-4257-8a83-cd7693e76fe4","Type":"ContainerDied","Data":"53f7aa225097a44833451dab881a0aa0702ecf812f922daa93786cc129b15abc"} Jun 02 04:45:54.074594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:54.074431 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dfd2sq" Jun 02 04:45:54.074594 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:54.074432 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53f7aa225097a44833451dab881a0aa0702ecf812f922daa93786cc129b15abc" Jun 02 04:45:57.896580 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896541 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt"] Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896822 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="util" Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896837 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="util" Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896850 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="extract" Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896856 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="extract" Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896863 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="pull" Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896868 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="pull" Jun 02 04:45:57.897036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.896926 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="6bdf4358-0000-4257-8a83-cd7693e76fe4" containerName="extract" Jun 02 04:45:57.943455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.943414 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt"] Jun 02 04:45:57.943618 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.943540 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:57.946173 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.946132 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 02 04:45:57.946394 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.946379 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7wzbz\"" Jun 02 04:45:57.946626 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.946606 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 02 04:45:57.957996 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.957967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6vlb\" (UniqueName: \"kubernetes.io/projected/b44f8abb-a306-4885-bf5a-028b0febcf21-kube-api-access-g6vlb\") pod \"cert-manager-operator-controller-manager-54b9655956-8xzqt\" (UID: \"b44f8abb-a306-4885-bf5a-028b0febcf21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:57.958112 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:57.958031 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b44f8abb-a306-4885-bf5a-028b0febcf21-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-8xzqt\" (UID: \"b44f8abb-a306-4885-bf5a-028b0febcf21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:58.058810 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.058778 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g6vlb\" (UniqueName: \"kubernetes.io/projected/b44f8abb-a306-4885-bf5a-028b0febcf21-kube-api-access-g6vlb\") pod \"cert-manager-operator-controller-manager-54b9655956-8xzqt\" (UID: \"b44f8abb-a306-4885-bf5a-028b0febcf21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:58.058934 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.058871 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b44f8abb-a306-4885-bf5a-028b0febcf21-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-8xzqt\" (UID: \"b44f8abb-a306-4885-bf5a-028b0febcf21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:58.059211 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.059194 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b44f8abb-a306-4885-bf5a-028b0febcf21-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-8xzqt\" (UID: \"b44f8abb-a306-4885-bf5a-028b0febcf21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:58.069277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.069244 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 02 04:45:58.079565 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.079545 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 02 04:45:58.089491 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.089467 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6vlb\" (UniqueName: \"kubernetes.io/projected/b44f8abb-a306-4885-bf5a-028b0febcf21-kube-api-access-g6vlb\") pod \"cert-manager-operator-controller-manager-54b9655956-8xzqt\" (UID: \"b44f8abb-a306-4885-bf5a-028b0febcf21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:58.254829 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.254747 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7wzbz\"" Jun 02 04:45:58.262960 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.262937 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" Jun 02 04:45:58.416306 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:58.416267 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt"] Jun 02 04:45:58.419888 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:45:58.419858 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44f8abb_a306_4885_bf5a_028b0febcf21.slice/crio-a759abd19155155cb229e8bab970b9ee51a479117a6d8db50cd6dc299aaeb850 WatchSource:0}: Error finding container a759abd19155155cb229e8bab970b9ee51a479117a6d8db50cd6dc299aaeb850: Status 404 returned error can't find the container with id a759abd19155155cb229e8bab970b9ee51a479117a6d8db50cd6dc299aaeb850 Jun 02 04:45:59.088299 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:45:59.088264 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" event={"ID":"b44f8abb-a306-4885-bf5a-028b0febcf21","Type":"ContainerStarted","Data":"a759abd19155155cb229e8bab970b9ee51a479117a6d8db50cd6dc299aaeb850"} Jun 02 04:46:01.096652 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.096611 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" event={"ID":"b44f8abb-a306-4885-bf5a-028b0febcf21","Type":"ContainerStarted","Data":"5fac6ca47bd719c58cb0ab166b7f633226ada928d6b33c47c85119f485349cc3"} Jun 02 04:46:01.120048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.119998 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-8xzqt" podStartSLOduration=2.315897346 podStartE2EDuration="4.119983852s" podCreationTimestamp="2026-06-02 04:45:57 +0000 UTC" firstStartedPulling="2026-06-02 04:45:58.42255862 +0000 UTC m=+240.846328874" lastFinishedPulling="2026-06-02 04:46:00.226645122 +0000 UTC m=+242.650415380" observedRunningTime="2026-06-02 04:46:01.118909978 +0000 UTC m=+243.542680252" watchObservedRunningTime="2026-06-02 04:46:01.119983852 +0000 UTC m=+243.543754115" Jun 02 04:46:01.864264 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.864219 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6"] Jun 02 04:46:01.866820 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.866795 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.869262 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.869235 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7hkf\"" Jun 02 04:46:01.869401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.869380 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 04:46:01.869472 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.869451 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 04:46:01.885310 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.885274 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6"] Jun 02 04:46:01.887808 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.887781 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz7n8\" (UniqueName: \"kubernetes.io/projected/4cb706df-9e04-4354-9aea-37664cf60ba4-kube-api-access-kz7n8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.887949 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.887826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.887949 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.887850 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.988948 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.988909 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kz7n8\" (UniqueName: \"kubernetes.io/projected/4cb706df-9e04-4354-9aea-37664cf60ba4-kube-api-access-kz7n8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.989123 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.988957 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.989123 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.988978 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.989359 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.989339 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:01.989396 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:01.989373 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:02.000707 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:02.000669 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz7n8\" (UniqueName: \"kubernetes.io/projected/4cb706df-9e04-4354-9aea-37664cf60ba4-kube-api-access-kz7n8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:02.175733 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:02.175643 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:02.322619 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:02.322580 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6"] Jun 02 04:46:02.325458 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:46:02.325426 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb706df_9e04_4354_9aea_37664cf60ba4.slice/crio-7b3731584df7ea38c54e2c4d84f5ef80960eddd240c5c3d29af84839bb6a9adf WatchSource:0}: Error finding container 7b3731584df7ea38c54e2c4d84f5ef80960eddd240c5c3d29af84839bb6a9adf: Status 404 returned error can't find the container with id 7b3731584df7ea38c54e2c4d84f5ef80960eddd240c5c3d29af84839bb6a9adf Jun 02 04:46:03.103652 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:03.103618 2566 generic.go:358] "Generic (PLEG): container finished" podID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerID="aa0a37e659805fd064ba2dac0d872e7b0858fc96974db7a09869ea045f047824" exitCode=0 Jun 02 04:46:03.103835 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:03.103675 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" event={"ID":"4cb706df-9e04-4354-9aea-37664cf60ba4","Type":"ContainerDied","Data":"aa0a37e659805fd064ba2dac0d872e7b0858fc96974db7a09869ea045f047824"} Jun 02 04:46:03.103835 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:03.103701 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" event={"ID":"4cb706df-9e04-4354-9aea-37664cf60ba4","Type":"ContainerStarted","Data":"7b3731584df7ea38c54e2c4d84f5ef80960eddd240c5c3d29af84839bb6a9adf"} Jun 02 04:46:06.119625 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.119538 2566 generic.go:358] "Generic (PLEG): container finished" podID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerID="df3b0813f63cf74b2a46db7a0c5a78aecaf3474bbf3351d9af5d197dd4f48c16" exitCode=0 Jun 02 04:46:06.120005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.119616 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" event={"ID":"4cb706df-9e04-4354-9aea-37664cf60ba4","Type":"ContainerDied","Data":"df3b0813f63cf74b2a46db7a0c5a78aecaf3474bbf3351d9af5d197dd4f48c16"} Jun 02 04:46:06.785809 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.785771 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-rm2js"] Jun 02 04:46:06.787979 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.787954 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:06.790808 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.790784 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 02 04:46:06.791483 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.791467 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 02 04:46:06.791541 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.791494 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5ncgp\"" Jun 02 04:46:06.804235 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.804208 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-rm2js"] Jun 02 04:46:06.833482 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.833443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhn2\" (UniqueName: \"kubernetes.io/projected/f39aaf66-0944-4c52-8ff1-12b970c98c26-kube-api-access-xjhn2\") pod \"cert-manager-cainjector-68b757865b-rm2js\" (UID: \"f39aaf66-0944-4c52-8ff1-12b970c98c26\") " pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:06.833693 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.833511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f39aaf66-0944-4c52-8ff1-12b970c98c26-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-rm2js\" (UID: \"f39aaf66-0944-4c52-8ff1-12b970c98c26\") " pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:06.934816 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.934782 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhn2\" (UniqueName: \"kubernetes.io/projected/f39aaf66-0944-4c52-8ff1-12b970c98c26-kube-api-access-xjhn2\") pod \"cert-manager-cainjector-68b757865b-rm2js\" (UID: \"f39aaf66-0944-4c52-8ff1-12b970c98c26\") " pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:06.934987 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.934844 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f39aaf66-0944-4c52-8ff1-12b970c98c26-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-rm2js\" (UID: \"f39aaf66-0944-4c52-8ff1-12b970c98c26\") " pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:06.945959 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.945936 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f39aaf66-0944-4c52-8ff1-12b970c98c26-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-rm2js\" (UID: \"f39aaf66-0944-4c52-8ff1-12b970c98c26\") " pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:06.946189 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:06.946168 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhn2\" (UniqueName: \"kubernetes.io/projected/f39aaf66-0944-4c52-8ff1-12b970c98c26-kube-api-access-xjhn2\") pod \"cert-manager-cainjector-68b757865b-rm2js\" (UID: \"f39aaf66-0944-4c52-8ff1-12b970c98c26\") " pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:07.110785 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:07.110742 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" Jun 02 04:46:07.124966 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:07.124933 2566 generic.go:358] "Generic (PLEG): container finished" podID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerID="61fa8e16f643d92332e1e4b98fcabf04a18057ee6c0c5a0b6ea24880f45afd0c" exitCode=0 Jun 02 04:46:07.125344 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:07.125020 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" event={"ID":"4cb706df-9e04-4354-9aea-37664cf60ba4","Type":"ContainerDied","Data":"61fa8e16f643d92332e1e4b98fcabf04a18057ee6c0c5a0b6ea24880f45afd0c"} Jun 02 04:46:07.240581 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:07.240512 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-rm2js"] Jun 02 04:46:07.242756 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:46:07.242726 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf39aaf66_0944_4c52_8ff1_12b970c98c26.slice/crio-39e07c3923ceb70fb683eaceb4812bb7ffcf7eb378c4ef956b28f45fff980c13 WatchSource:0}: Error finding container 39e07c3923ceb70fb683eaceb4812bb7ffcf7eb378c4ef956b28f45fff980c13: Status 404 returned error can't find the container with id 39e07c3923ceb70fb683eaceb4812bb7ffcf7eb378c4ef956b28f45fff980c13 Jun 02 04:46:08.128201 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.128166 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" event={"ID":"f39aaf66-0944-4c52-8ff1-12b970c98c26","Type":"ContainerStarted","Data":"39e07c3923ceb70fb683eaceb4812bb7ffcf7eb378c4ef956b28f45fff980c13"} Jun 02 04:46:08.249271 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.249246 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:08.347354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.347310 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-bundle\") pod \"4cb706df-9e04-4354-9aea-37664cf60ba4\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " Jun 02 04:46:08.347549 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.347365 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-util\") pod \"4cb706df-9e04-4354-9aea-37664cf60ba4\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " Jun 02 04:46:08.347549 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.347437 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz7n8\" (UniqueName: \"kubernetes.io/projected/4cb706df-9e04-4354-9aea-37664cf60ba4-kube-api-access-kz7n8\") pod \"4cb706df-9e04-4354-9aea-37664cf60ba4\" (UID: \"4cb706df-9e04-4354-9aea-37664cf60ba4\") " Jun 02 04:46:08.347733 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.347709 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-bundle" (OuterVolumeSpecName: "bundle") pod "4cb706df-9e04-4354-9aea-37664cf60ba4" (UID: "4cb706df-9e04-4354-9aea-37664cf60ba4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:46:08.350454 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.350420 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb706df-9e04-4354-9aea-37664cf60ba4-kube-api-access-kz7n8" (OuterVolumeSpecName: "kube-api-access-kz7n8") pod "4cb706df-9e04-4354-9aea-37664cf60ba4" (UID: "4cb706df-9e04-4354-9aea-37664cf60ba4"). InnerVolumeSpecName "kube-api-access-kz7n8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:46:08.352004 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.351982 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-util" (OuterVolumeSpecName: "util") pod "4cb706df-9e04-4354-9aea-37664cf60ba4" (UID: "4cb706df-9e04-4354-9aea-37664cf60ba4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:46:08.448057 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.447969 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz7n8\" (UniqueName: \"kubernetes.io/projected/4cb706df-9e04-4354-9aea-37664cf60ba4-kube-api-access-kz7n8\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:46:08.448057 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.448000 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:46:08.448057 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:08.448011 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cb706df-9e04-4354-9aea-37664cf60ba4-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:46:09.133948 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:09.133889 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" event={"ID":"4cb706df-9e04-4354-9aea-37664cf60ba4","Type":"ContainerDied","Data":"7b3731584df7ea38c54e2c4d84f5ef80960eddd240c5c3d29af84839bb6a9adf"} Jun 02 04:46:09.133948 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:09.133931 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b3731584df7ea38c54e2c4d84f5ef80960eddd240c5c3d29af84839bb6a9adf" Jun 02 04:46:09.134500 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:09.133967 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgw2l6" Jun 02 04:46:11.142292 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:11.142258 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" event={"ID":"f39aaf66-0944-4c52-8ff1-12b970c98c26","Type":"ContainerStarted","Data":"b661cf4a0ef13e0220eebca9d6c2e5d5012c6aff7a4d85255e21cfa80c9cb474"} Jun 02 04:46:11.159691 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:11.159635 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-rm2js" podStartSLOduration=2.040567169 podStartE2EDuration="5.159619092s" podCreationTimestamp="2026-06-02 04:46:06 +0000 UTC" firstStartedPulling="2026-06-02 04:46:07.244740399 +0000 UTC m=+249.668510642" lastFinishedPulling="2026-06-02 04:46:10.363792324 +0000 UTC m=+252.787562565" observedRunningTime="2026-06-02 04:46:11.158292425 +0000 UTC m=+253.582062688" watchObservedRunningTime="2026-06-02 04:46:11.159619092 +0000 UTC m=+253.583389354" Jun 02 04:46:18.467766 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.467727 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv"] Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468022 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="pull" Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468033 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="pull" Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468048 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="util" Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468053 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="util" Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468062 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="extract" Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468067 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="extract" Jun 02 04:46:18.468164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.468116 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="4cb706df-9e04-4354-9aea-37664cf60ba4" containerName="extract" Jun 02 04:46:18.469984 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.469967 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.472248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.472223 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 04:46:18.473065 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.473036 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 04:46:18.473188 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.473109 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-k7fbv\"" Jun 02 04:46:18.482402 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.482374 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv"] Jun 02 04:46:18.535536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.535493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gm86\" (UniqueName: \"kubernetes.io/projected/cdd319e3-1f35-4b03-b8bd-ccdf327897dc-kube-api-access-2gm86\") pod \"openshift-lws-operator-bfc7f696d-nbrkv\" (UID: \"cdd319e3-1f35-4b03-b8bd-ccdf327897dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.535695 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.535558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdd319e3-1f35-4b03-b8bd-ccdf327897dc-tmp\") pod \"openshift-lws-operator-bfc7f696d-nbrkv\" (UID: \"cdd319e3-1f35-4b03-b8bd-ccdf327897dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.636157 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.636105 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2gm86\" (UniqueName: \"kubernetes.io/projected/cdd319e3-1f35-4b03-b8bd-ccdf327897dc-kube-api-access-2gm86\") pod \"openshift-lws-operator-bfc7f696d-nbrkv\" (UID: \"cdd319e3-1f35-4b03-b8bd-ccdf327897dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.636327 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.636200 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdd319e3-1f35-4b03-b8bd-ccdf327897dc-tmp\") pod \"openshift-lws-operator-bfc7f696d-nbrkv\" (UID: \"cdd319e3-1f35-4b03-b8bd-ccdf327897dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.636552 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.636530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdd319e3-1f35-4b03-b8bd-ccdf327897dc-tmp\") pod \"openshift-lws-operator-bfc7f696d-nbrkv\" (UID: \"cdd319e3-1f35-4b03-b8bd-ccdf327897dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.647825 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.647789 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gm86\" (UniqueName: \"kubernetes.io/projected/cdd319e3-1f35-4b03-b8bd-ccdf327897dc-kube-api-access-2gm86\") pod \"openshift-lws-operator-bfc7f696d-nbrkv\" (UID: \"cdd319e3-1f35-4b03-b8bd-ccdf327897dc\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.778909 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.778790 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" Jun 02 04:46:18.904786 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:18.904759 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv"] Jun 02 04:46:18.907601 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:46:18.907576 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdd319e3_1f35_4b03_b8bd_ccdf327897dc.slice/crio-3f5324152585e7f6715c97360334b143734425995ee7bd3bbfe6b74f321a028c WatchSource:0}: Error finding container 3f5324152585e7f6715c97360334b143734425995ee7bd3bbfe6b74f321a028c: Status 404 returned error can't find the container with id 3f5324152585e7f6715c97360334b143734425995ee7bd3bbfe6b74f321a028c Jun 02 04:46:19.166774 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:19.166737 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" event={"ID":"cdd319e3-1f35-4b03-b8bd-ccdf327897dc","Type":"ContainerStarted","Data":"3f5324152585e7f6715c97360334b143734425995ee7bd3bbfe6b74f321a028c"} Jun 02 04:46:22.177113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:22.177075 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" event={"ID":"cdd319e3-1f35-4b03-b8bd-ccdf327897dc","Type":"ContainerStarted","Data":"5cf22a6a6a8590636e2d6275cc75b162a761e2ef31150e8d178ec4349f344881"} Jun 02 04:46:22.195744 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:22.195678 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-nbrkv" podStartSLOduration=1.939600445 podStartE2EDuration="4.195656979s" podCreationTimestamp="2026-06-02 04:46:18 +0000 UTC" firstStartedPulling="2026-06-02 04:46:18.909494446 +0000 UTC m=+261.333264687" lastFinishedPulling="2026-06-02 04:46:21.16555098 +0000 UTC m=+263.589321221" observedRunningTime="2026-06-02 04:46:22.194787122 +0000 UTC m=+264.618557382" watchObservedRunningTime="2026-06-02 04:46:22.195656979 +0000 UTC m=+264.619427246" Jun 02 04:46:25.507554 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.507522 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5"] Jun 02 04:46:25.509955 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.509936 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.512236 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.512203 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 04:46:25.512965 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.512946 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 04:46:25.513052 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.513014 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7hkf\"" Jun 02 04:46:25.519115 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.519089 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5"] Jun 02 04:46:25.596729 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.596690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.596729 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.596736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpqpm\" (UniqueName: \"kubernetes.io/projected/8141dd81-c26d-444f-8c33-9431ccd7f699-kube-api-access-jpqpm\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.596951 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.596841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.697825 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.697788 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.697993 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.697859 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.697993 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.697881 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jpqpm\" (UniqueName: \"kubernetes.io/projected/8141dd81-c26d-444f-8c33-9431ccd7f699-kube-api-access-jpqpm\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.698180 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.698157 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.698277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.698253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.706730 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.706704 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpqpm\" (UniqueName: \"kubernetes.io/projected/8141dd81-c26d-444f-8c33-9431ccd7f699-kube-api-access-jpqpm\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.819504 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.819471 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:25.948427 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:25.948389 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5"] Jun 02 04:46:25.951095 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:46:25.951064 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8141dd81_c26d_444f_8c33_9431ccd7f699.slice/crio-c0af5382bd904d66abb5c2ddd62f140751039ec19f02217d2df027c7f92c6a42 WatchSource:0}: Error finding container c0af5382bd904d66abb5c2ddd62f140751039ec19f02217d2df027c7f92c6a42: Status 404 returned error can't find the container with id c0af5382bd904d66abb5c2ddd62f140751039ec19f02217d2df027c7f92c6a42 Jun 02 04:46:26.190588 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:26.190499 2566 generic.go:358] "Generic (PLEG): container finished" podID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerID="c36c906edf9a9163882e0c4ff4a6e2cda00dd98a0b93c77c780bc29614c9a01f" exitCode=0 Jun 02 04:46:26.190728 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:26.190584 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" event={"ID":"8141dd81-c26d-444f-8c33-9431ccd7f699","Type":"ContainerDied","Data":"c36c906edf9a9163882e0c4ff4a6e2cda00dd98a0b93c77c780bc29614c9a01f"} Jun 02 04:46:26.190728 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:26.190619 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" event={"ID":"8141dd81-c26d-444f-8c33-9431ccd7f699","Type":"ContainerStarted","Data":"c0af5382bd904d66abb5c2ddd62f140751039ec19f02217d2df027c7f92c6a42"} Jun 02 04:46:27.195619 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:27.195581 2566 generic.go:358] "Generic (PLEG): container finished" podID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerID="78425883c8fba8cec1f847e22121216268549b0165b4218e7678592acd9284d5" exitCode=0 Jun 02 04:46:27.196102 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:27.195633 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" event={"ID":"8141dd81-c26d-444f-8c33-9431ccd7f699","Type":"ContainerDied","Data":"78425883c8fba8cec1f847e22121216268549b0165b4218e7678592acd9284d5"} Jun 02 04:46:28.200855 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:28.200817 2566 generic.go:358] "Generic (PLEG): container finished" podID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerID="7280d1971c2fdac06f4a0624305ee14f446d1b9b6c8fd0021f717239a16315af" exitCode=0 Jun 02 04:46:28.201260 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:28.200900 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" event={"ID":"8141dd81-c26d-444f-8c33-9431ccd7f699","Type":"ContainerDied","Data":"7280d1971c2fdac06f4a0624305ee14f446d1b9b6c8fd0021f717239a16315af"} Jun 02 04:46:29.329920 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.329888 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:29.432160 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.432100 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-util\") pod \"8141dd81-c26d-444f-8c33-9431ccd7f699\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " Jun 02 04:46:29.432354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.432193 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpqpm\" (UniqueName: \"kubernetes.io/projected/8141dd81-c26d-444f-8c33-9431ccd7f699-kube-api-access-jpqpm\") pod \"8141dd81-c26d-444f-8c33-9431ccd7f699\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " Jun 02 04:46:29.432354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.432228 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-bundle\") pod \"8141dd81-c26d-444f-8c33-9431ccd7f699\" (UID: \"8141dd81-c26d-444f-8c33-9431ccd7f699\") " Jun 02 04:46:29.433010 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.432964 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-bundle" (OuterVolumeSpecName: "bundle") pod "8141dd81-c26d-444f-8c33-9431ccd7f699" (UID: "8141dd81-c26d-444f-8c33-9431ccd7f699"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:46:29.434414 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.434384 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8141dd81-c26d-444f-8c33-9431ccd7f699-kube-api-access-jpqpm" (OuterVolumeSpecName: "kube-api-access-jpqpm") pod "8141dd81-c26d-444f-8c33-9431ccd7f699" (UID: "8141dd81-c26d-444f-8c33-9431ccd7f699"). InnerVolumeSpecName "kube-api-access-jpqpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:46:29.440149 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.440100 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-util" (OuterVolumeSpecName: "util") pod "8141dd81-c26d-444f-8c33-9431ccd7f699" (UID: "8141dd81-c26d-444f-8c33-9431ccd7f699"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:46:29.533775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.533671 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpqpm\" (UniqueName: \"kubernetes.io/projected/8141dd81-c26d-444f-8c33-9431ccd7f699-kube-api-access-jpqpm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:46:29.533775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.533721 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:46:29.533775 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:29.533731 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8141dd81-c26d-444f-8c33-9431ccd7f699-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:46:30.209182 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:30.209123 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" event={"ID":"8141dd81-c26d-444f-8c33-9431ccd7f699","Type":"ContainerDied","Data":"c0af5382bd904d66abb5c2ddd62f140751039ec19f02217d2df027c7f92c6a42"} Jun 02 04:46:30.209182 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:30.209173 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cddnrh5" Jun 02 04:46:30.209182 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:30.209182 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0af5382bd904d66abb5c2ddd62f140751039ec19f02217d2df027c7f92c6a42" Jun 02 04:46:42.280243 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280207 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr"] Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280487 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="pull" Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280497 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="pull" Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280508 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="util" Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280515 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="util" Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280530 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="extract" Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280536 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="extract" Jun 02 04:46:42.280659 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.280583 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="8141dd81-c26d-444f-8c33-9431ccd7f699" containerName="extract" Jun 02 04:46:42.284824 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.284806 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.288953 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.288928 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 04:46:42.289692 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.289673 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 04:46:42.289846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.289805 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6bclf\"" Jun 02 04:46:42.290805 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.290787 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 04:46:42.291967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.291670 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 04:46:42.305628 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.305590 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr"] Jun 02 04:46:42.337443 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.337412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/861a43e3-0d05-4eb0-bd13-27d788a85c6d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.337610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.337457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/861a43e3-0d05-4eb0-bd13-27d788a85c6d-manifests\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.337610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.337489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n68jm\" (UniqueName: \"kubernetes.io/projected/861a43e3-0d05-4eb0-bd13-27d788a85c6d-kube-api-access-n68jm\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.337610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.337513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/861a43e3-0d05-4eb0-bd13-27d788a85c6d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.337610 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.337531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/861a43e3-0d05-4eb0-bd13-27d788a85c6d-tmp\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.438121 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.438067 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/861a43e3-0d05-4eb0-bd13-27d788a85c6d-manifests\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.438121 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.438124 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n68jm\" (UniqueName: \"kubernetes.io/projected/861a43e3-0d05-4eb0-bd13-27d788a85c6d-kube-api-access-n68jm\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.438421 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.438174 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/861a43e3-0d05-4eb0-bd13-27d788a85c6d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.438421 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.438192 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/861a43e3-0d05-4eb0-bd13-27d788a85c6d-tmp\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.438421 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.438255 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/861a43e3-0d05-4eb0-bd13-27d788a85c6d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.438579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.438555 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/861a43e3-0d05-4eb0-bd13-27d788a85c6d-manifests\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.440578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.440556 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/861a43e3-0d05-4eb0-bd13-27d788a85c6d-tmp\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.440705 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.440688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/861a43e3-0d05-4eb0-bd13-27d788a85c6d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.440879 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.440857 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/861a43e3-0d05-4eb0-bd13-27d788a85c6d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.448711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.448683 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n68jm\" (UniqueName: \"kubernetes.io/projected/861a43e3-0d05-4eb0-bd13-27d788a85c6d-kube-api-access-n68jm\") pod \"opendatahub-operator-controller-manager-6b5b979d7-bpzjr\" (UID: \"861a43e3-0d05-4eb0-bd13-27d788a85c6d\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.595262 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.595227 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:42.737472 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:42.737437 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr"] Jun 02 04:46:42.740468 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:46:42.740436 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861a43e3_0d05_4eb0_bd13_27d788a85c6d.slice/crio-38228e89e6fcf069c8b9942d12a334e5634a471eede5cf716c544b4061ceeb10 WatchSource:0}: Error finding container 38228e89e6fcf069c8b9942d12a334e5634a471eede5cf716c544b4061ceeb10: Status 404 returned error can't find the container with id 38228e89e6fcf069c8b9942d12a334e5634a471eede5cf716c544b4061ceeb10 Jun 02 04:46:43.249683 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:43.249643 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" event={"ID":"861a43e3-0d05-4eb0-bd13-27d788a85c6d","Type":"ContainerStarted","Data":"38228e89e6fcf069c8b9942d12a334e5634a471eede5cf716c544b4061ceeb10"} Jun 02 04:46:46.264282 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:46.264237 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" event={"ID":"861a43e3-0d05-4eb0-bd13-27d788a85c6d","Type":"ContainerStarted","Data":"60d98ef32088713f26219f701ae45a4b49b4df0948fd08745771a32e9bdb4ab1"} Jun 02 04:46:47.268834 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:47.268795 2566 generic.go:358] "Generic (PLEG): container finished" podID="861a43e3-0d05-4eb0-bd13-27d788a85c6d" containerID="60d98ef32088713f26219f701ae45a4b49b4df0948fd08745771a32e9bdb4ab1" exitCode=0 Jun 02 04:46:47.269235 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:47.268858 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" event={"ID":"861a43e3-0d05-4eb0-bd13-27d788a85c6d","Type":"ContainerDied","Data":"60d98ef32088713f26219f701ae45a4b49b4df0948fd08745771a32e9bdb4ab1"} Jun 02 04:46:48.273193 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:48.273154 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" event={"ID":"861a43e3-0d05-4eb0-bd13-27d788a85c6d","Type":"ContainerStarted","Data":"b7d7267ba0c0ab3280590b63144a2fa1487d994c35028d025150ee936e86c20f"} Jun 02 04:46:48.273658 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:48.273221 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:46:48.297844 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:48.297787 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" podStartSLOduration=1.50190249 podStartE2EDuration="6.297771686s" podCreationTimestamp="2026-06-02 04:46:42 +0000 UTC" firstStartedPulling="2026-06-02 04:46:42.742245141 +0000 UTC m=+285.166015382" lastFinishedPulling="2026-06-02 04:46:47.538114336 +0000 UTC m=+289.961884578" observedRunningTime="2026-06-02 04:46:48.295854009 +0000 UTC m=+290.719624277" watchObservedRunningTime="2026-06-02 04:46:48.297771686 +0000 UTC m=+290.721541949" Jun 02 04:46:58.047462 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:58.047428 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:46:58.048251 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:58.048230 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:46:58.051964 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:58.051934 2566 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 04:46:59.278923 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:46:59.278892 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b5b979d7-bpzjr" Jun 02 04:47:01.910157 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:01.910104 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-89j4n"] Jun 02 04:47:01.913440 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:01.913418 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:01.916170 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:01.916134 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 02 04:47:01.916389 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:01.916369 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lpbkx\"" Jun 02 04:47:01.935814 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:01.935782 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-89j4n"] Jun 02 04:47:02.000657 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.000623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grhs4\" (UniqueName: \"kubernetes.io/projected/30b6ba1a-de1a-4563-8c9c-55314dfd2977-kube-api-access-grhs4\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:02.000834 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.000678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:02.089624 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.089585 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65977b746f-vdjwv"] Jun 02 04:47:02.095250 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.095202 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.099296 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.099273 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 04:47:02.100102 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.100077 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 04:47:02.100268 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.100243 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 04:47:02.100377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.100244 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6hdl4\"" Jun 02 04:47:02.100622 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.100606 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 04:47:02.101201 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.101180 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-grhs4\" (UniqueName: \"kubernetes.io/projected/30b6ba1a-de1a-4563-8c9c-55314dfd2977-kube-api-access-grhs4\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:02.101269 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.101254 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:02.101372 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:47:02.101360 2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 02 04:47:02.101419 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:47:02.101410 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert podName:30b6ba1a-de1a-4563-8c9c-55314dfd2977 nodeName:}" failed. No retries permitted until 2026-06-02 04:47:02.601391766 +0000 UTC m=+305.025162020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert") pod "odh-model-controller-76588b95-89j4n" (UID: "30b6ba1a-de1a-4563-8c9c-55314dfd2977") : secret "odh-model-controller-webhook-cert" not found Jun 02 04:47:02.113678 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.113647 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65977b746f-vdjwv"] Jun 02 04:47:02.132225 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.132191 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grhs4\" (UniqueName: \"kubernetes.io/projected/30b6ba1a-de1a-4563-8c9c-55314dfd2977-kube-api-access-grhs4\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:02.201925 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.201818 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f4cddb8-a86e-41f7-b747-8e6be4887e66-tmp\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.201925 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.201899 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q6zn\" (UniqueName: \"kubernetes.io/projected/8f4cddb8-a86e-41f7-b747-8e6be4887e66-kube-api-access-5q6zn\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.202265 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.201946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f4cddb8-a86e-41f7-b747-8e6be4887e66-tls-certs\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.303445 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.303405 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f4cddb8-a86e-41f7-b747-8e6be4887e66-tls-certs\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.303650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.303587 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f4cddb8-a86e-41f7-b747-8e6be4887e66-tmp\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.303728 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.303683 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5q6zn\" (UniqueName: \"kubernetes.io/projected/8f4cddb8-a86e-41f7-b747-8e6be4887e66-kube-api-access-5q6zn\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.305870 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.305839 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f4cddb8-a86e-41f7-b747-8e6be4887e66-tmp\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.306107 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.306088 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f4cddb8-a86e-41f7-b747-8e6be4887e66-tls-certs\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.321007 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.320976 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q6zn\" (UniqueName: \"kubernetes.io/projected/8f4cddb8-a86e-41f7-b747-8e6be4887e66-kube-api-access-5q6zn\") pod \"kube-auth-proxy-65977b746f-vdjwv\" (UID: \"8f4cddb8-a86e-41f7-b747-8e6be4887e66\") " pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.405967 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.405927 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" Jun 02 04:47:02.577925 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.577895 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65977b746f-vdjwv"] Jun 02 04:47:02.580290 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:47:02.580260 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4cddb8_a86e_41f7_b747_8e6be4887e66.slice/crio-82f88b2de49f8f86efdaebc00fde02a1fdfe40a8f91cbc74676005168d32b8b8 WatchSource:0}: Error finding container 82f88b2de49f8f86efdaebc00fde02a1fdfe40a8f91cbc74676005168d32b8b8: Status 404 returned error can't find the container with id 82f88b2de49f8f86efdaebc00fde02a1fdfe40a8f91cbc74676005168d32b8b8 Jun 02 04:47:02.582104 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.582089 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 04:47:02.606608 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:02.606578 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:02.606780 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:47:02.606741 2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 02 04:47:02.606847 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:47:02.606821 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert podName:30b6ba1a-de1a-4563-8c9c-55314dfd2977 nodeName:}" failed. No retries permitted until 2026-06-02 04:47:03.60679848 +0000 UTC m=+306.030568720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert") pod "odh-model-controller-76588b95-89j4n" (UID: "30b6ba1a-de1a-4563-8c9c-55314dfd2977") : secret "odh-model-controller-webhook-cert" not found Jun 02 04:47:03.049017 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.048981 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-sjb96"] Jun 02 04:47:03.053630 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.053611 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.057206 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.057185 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 02 04:47:03.057313 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.057186 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-w7wtf\"" Jun 02 04:47:03.069842 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.069816 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-sjb96"] Jun 02 04:47:03.111626 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.111585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g47wz\" (UniqueName: \"kubernetes.io/projected/bcb9f931-ede4-494d-9217-4b60393e563e-kube-api-access-g47wz\") pod \"model-serving-api-6577596cd5-sjb96\" (UID: \"bcb9f931-ede4-494d-9217-4b60393e563e\") " pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.111813 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.111678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcb9f931-ede4-494d-9217-4b60393e563e-tls-certs\") pod \"model-serving-api-6577596cd5-sjb96\" (UID: \"bcb9f931-ede4-494d-9217-4b60393e563e\") " pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.212605 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.212561 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcb9f931-ede4-494d-9217-4b60393e563e-tls-certs\") pod \"model-serving-api-6577596cd5-sjb96\" (UID: \"bcb9f931-ede4-494d-9217-4b60393e563e\") " pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.212788 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.212687 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g47wz\" (UniqueName: \"kubernetes.io/projected/bcb9f931-ede4-494d-9217-4b60393e563e-kube-api-access-g47wz\") pod \"model-serving-api-6577596cd5-sjb96\" (UID: \"bcb9f931-ede4-494d-9217-4b60393e563e\") " pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.215935 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.215883 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcb9f931-ede4-494d-9217-4b60393e563e-tls-certs\") pod \"model-serving-api-6577596cd5-sjb96\" (UID: \"bcb9f931-ede4-494d-9217-4b60393e563e\") " pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.225675 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.225646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g47wz\" (UniqueName: \"kubernetes.io/projected/bcb9f931-ede4-494d-9217-4b60393e563e-kube-api-access-g47wz\") pod \"model-serving-api-6577596cd5-sjb96\" (UID: \"bcb9f931-ede4-494d-9217-4b60393e563e\") " pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.322297 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.322255 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" event={"ID":"8f4cddb8-a86e-41f7-b747-8e6be4887e66","Type":"ContainerStarted","Data":"82f88b2de49f8f86efdaebc00fde02a1fdfe40a8f91cbc74676005168d32b8b8"} Jun 02 04:47:03.366502 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.366443 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:03.527464 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.527176 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-sjb96"] Jun 02 04:47:03.531082 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:47:03.530962 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb9f931_ede4_494d_9217_4b60393e563e.slice/crio-88a7aea170bb7477ae25bdf5031049bff36b2c9ff202e40c1b95add560e5c2da WatchSource:0}: Error finding container 88a7aea170bb7477ae25bdf5031049bff36b2c9ff202e40c1b95add560e5c2da: Status 404 returned error can't find the container with id 88a7aea170bb7477ae25bdf5031049bff36b2c9ff202e40c1b95add560e5c2da Jun 02 04:47:03.617249 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.617160 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:03.620347 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.620322 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30b6ba1a-de1a-4563-8c9c-55314dfd2977-cert\") pod \"odh-model-controller-76588b95-89j4n\" (UID: \"30b6ba1a-de1a-4563-8c9c-55314dfd2977\") " pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:03.698658 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.698624 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m"] Jun 02 04:47:03.703571 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.703547 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.706161 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.706121 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7hkf\"" Jun 02 04:47:03.706296 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.706163 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 04:47:03.706296 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.706173 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 04:47:03.713072 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.712600 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m"] Jun 02 04:47:03.722831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.722774 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:03.819262 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.819221 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwxjp\" (UniqueName: \"kubernetes.io/projected/d8d4ccde-ce9b-4df8-8330-6df14b68872d-kube-api-access-hwxjp\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.819428 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.819339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.819491 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.819459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.876330 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.876298 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-89j4n"] Jun 02 04:47:03.921216 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.921163 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hwxjp\" (UniqueName: \"kubernetes.io/projected/d8d4ccde-ce9b-4df8-8330-6df14b68872d-kube-api-access-hwxjp\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.921400 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.921238 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.921400 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.921318 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.921754 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.921724 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.921875 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.921779 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:03.934976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:03.934945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwxjp\" (UniqueName: \"kubernetes.io/projected/d8d4ccde-ce9b-4df8-8330-6df14b68872d-kube-api-access-hwxjp\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:04.015932 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:04.015891 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:04.329398 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:04.329353 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-sjb96" event={"ID":"bcb9f931-ede4-494d-9217-4b60393e563e","Type":"ContainerStarted","Data":"88a7aea170bb7477ae25bdf5031049bff36b2c9ff202e40c1b95add560e5c2da"} Jun 02 04:47:04.538959 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:47:04.538914 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b6ba1a_de1a_4563_8c9c_55314dfd2977.slice/crio-fcaa361dce945ef1fe21e99accdfeacd554b2b30244e1cc27675656239ef796b WatchSource:0}: Error finding container fcaa361dce945ef1fe21e99accdfeacd554b2b30244e1cc27675656239ef796b: Status 404 returned error can't find the container with id fcaa361dce945ef1fe21e99accdfeacd554b2b30244e1cc27675656239ef796b Jun 02 04:47:05.336847 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:05.336807 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-89j4n" event={"ID":"30b6ba1a-de1a-4563-8c9c-55314dfd2977","Type":"ContainerStarted","Data":"fcaa361dce945ef1fe21e99accdfeacd554b2b30244e1cc27675656239ef796b"} Jun 02 04:47:05.368200 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:05.367943 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m"] Jun 02 04:47:05.371096 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:47:05.371064 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d4ccde_ce9b_4df8_8330_6df14b68872d.slice/crio-e32d707c008312403c57b38acc7a19eabf091d2dac503f8c965f8569dd31eff9 WatchSource:0}: Error finding container e32d707c008312403c57b38acc7a19eabf091d2dac503f8c965f8569dd31eff9: Status 404 returned error can't find the container with id e32d707c008312403c57b38acc7a19eabf091d2dac503f8c965f8569dd31eff9 Jun 02 04:47:06.348436 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.348390 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" event={"ID":"8f4cddb8-a86e-41f7-b747-8e6be4887e66","Type":"ContainerStarted","Data":"c00aba6819e9b03b68644621e171670b51ddca226e3d0125fd30461dfce05a77"} Jun 02 04:47:06.350301 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.350266 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-sjb96" event={"ID":"bcb9f931-ede4-494d-9217-4b60393e563e","Type":"ContainerStarted","Data":"8fb060352523831b74848251a7b54c54abd429a1408c46b85f0664388f793886"} Jun 02 04:47:06.350443 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.350358 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:06.351825 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.351794 2566 generic.go:358] "Generic (PLEG): container finished" podID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerID="2e8dfd15a9ff58dc1946e6f5d7a5cea3d4192a0ab40ae79da631f423af03a469" exitCode=0 Jun 02 04:47:06.351917 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.351872 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" event={"ID":"d8d4ccde-ce9b-4df8-8330-6df14b68872d","Type":"ContainerDied","Data":"2e8dfd15a9ff58dc1946e6f5d7a5cea3d4192a0ab40ae79da631f423af03a469"} Jun 02 04:47:06.351917 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.351897 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" event={"ID":"d8d4ccde-ce9b-4df8-8330-6df14b68872d","Type":"ContainerStarted","Data":"e32d707c008312403c57b38acc7a19eabf091d2dac503f8c965f8569dd31eff9"} Jun 02 04:47:06.368862 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.368785 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65977b746f-vdjwv" podStartSLOduration=1.6682874760000002 podStartE2EDuration="4.368767042s" podCreationTimestamp="2026-06-02 04:47:02 +0000 UTC" firstStartedPulling="2026-06-02 04:47:02.582239854 +0000 UTC m=+305.006010098" lastFinishedPulling="2026-06-02 04:47:05.282719422 +0000 UTC m=+307.706489664" observedRunningTime="2026-06-02 04:47:06.367744436 +0000 UTC m=+308.791514700" watchObservedRunningTime="2026-06-02 04:47:06.368767042 +0000 UTC m=+308.792537304" Jun 02 04:47:06.408533 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:06.408475 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-sjb96" podStartSLOduration=1.07009515 podStartE2EDuration="3.408461705s" podCreationTimestamp="2026-06-02 04:47:03 +0000 UTC" firstStartedPulling="2026-06-02 04:47:03.533122694 +0000 UTC m=+305.956892949" lastFinishedPulling="2026-06-02 04:47:05.871489258 +0000 UTC m=+308.295259504" observedRunningTime="2026-06-02 04:47:06.406775244 +0000 UTC m=+308.830545508" watchObservedRunningTime="2026-06-02 04:47:06.408461705 +0000 UTC m=+308.832231968" Jun 02 04:47:08.361123 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:08.361039 2566 generic.go:358] "Generic (PLEG): container finished" podID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerID="5369b5c818873cdb244fb9cdaf0ea438a2fc7fb2a43afdd502808846e0a86f61" exitCode=0 Jun 02 04:47:08.361539 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:08.361120 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" event={"ID":"d8d4ccde-ce9b-4df8-8330-6df14b68872d","Type":"ContainerDied","Data":"5369b5c818873cdb244fb9cdaf0ea438a2fc7fb2a43afdd502808846e0a86f61"} Jun 02 04:47:08.362571 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:08.362544 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-89j4n" event={"ID":"30b6ba1a-de1a-4563-8c9c-55314dfd2977","Type":"ContainerStarted","Data":"b80e7b1c5a3b02589fbc6516f831bdcd7c4b1cf950f0c7b74e892f4b94cbb025"} Jun 02 04:47:08.362754 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:08.362651 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:08.396803 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:08.396746 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-89j4n" podStartSLOduration=4.084382663 podStartE2EDuration="7.396731616s" podCreationTimestamp="2026-06-02 04:47:01 +0000 UTC" firstStartedPulling="2026-06-02 04:47:04.54022315 +0000 UTC m=+306.963993394" lastFinishedPulling="2026-06-02 04:47:07.852572092 +0000 UTC m=+310.276342347" observedRunningTime="2026-06-02 04:47:08.394395848 +0000 UTC m=+310.818166111" watchObservedRunningTime="2026-06-02 04:47:08.396731616 +0000 UTC m=+310.820501878" Jun 02 04:47:09.367621 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.367585 2566 generic.go:358] "Generic (PLEG): container finished" podID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerID="3c1e40ad847bc3b7493ea8e6bd07962ea3ffed9278932dc96cb7d1ca2e4ccdaf" exitCode=0 Jun 02 04:47:09.368007 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.367662 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" event={"ID":"d8d4ccde-ce9b-4df8-8330-6df14b68872d","Type":"ContainerDied","Data":"3c1e40ad847bc3b7493ea8e6bd07962ea3ffed9278932dc96cb7d1ca2e4ccdaf"} Jun 02 04:47:09.426314 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.426271 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb"] Jun 02 04:47:09.429864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.429838 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.432470 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.432435 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 04:47:09.432710 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.432691 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-pgjpf\"" Jun 02 04:47:09.432785 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.432724 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 04:47:09.432785 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.432753 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 04:47:09.442679 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.442650 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb"] Jun 02 04:47:09.576916 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.576876 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlhgw\" (UniqueName: \"kubernetes.io/projected/a1764b75-cf72-4747-b57f-04f9a4b75e88-kube-api-access-xlhgw\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.577105 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.576951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a1764b75-cf72-4747-b57f-04f9a4b75e88-manager-config\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.577105 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.576978 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1764b75-cf72-4747-b57f-04f9a4b75e88-cert\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.577105 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.577063 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1764b75-cf72-4747-b57f-04f9a4b75e88-metrics-cert\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.677635 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.677544 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a1764b75-cf72-4747-b57f-04f9a4b75e88-manager-config\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.677635 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.677587 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1764b75-cf72-4747-b57f-04f9a4b75e88-cert\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.677846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.677729 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1764b75-cf72-4747-b57f-04f9a4b75e88-metrics-cert\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.677846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.677776 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhgw\" (UniqueName: \"kubernetes.io/projected/a1764b75-cf72-4747-b57f-04f9a4b75e88-kube-api-access-xlhgw\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.678434 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.678410 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a1764b75-cf72-4747-b57f-04f9a4b75e88-manager-config\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.680037 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.680012 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1764b75-cf72-4747-b57f-04f9a4b75e88-metrics-cert\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.680164 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.680046 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1764b75-cf72-4747-b57f-04f9a4b75e88-cert\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.689923 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.689896 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlhgw\" (UniqueName: \"kubernetes.io/projected/a1764b75-cf72-4747-b57f-04f9a4b75e88-kube-api-access-xlhgw\") pod \"lws-controller-manager-f8df7c74b-c9ngb\" (UID: \"a1764b75-cf72-4747-b57f-04f9a4b75e88\") " pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.739754 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.739714 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:09.886259 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:09.886219 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb"] Jun 02 04:47:09.889263 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:47:09.889239 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1764b75_cf72_4747_b57f_04f9a4b75e88.slice/crio-065408af969d1361013db58e7d7ef96b3a5fe19d9cd10e709c12d8f6c13bea48 WatchSource:0}: Error finding container 065408af969d1361013db58e7d7ef96b3a5fe19d9cd10e709c12d8f6c13bea48: Status 404 returned error can't find the container with id 065408af969d1361013db58e7d7ef96b3a5fe19d9cd10e709c12d8f6c13bea48 Jun 02 04:47:10.371682 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.371648 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" event={"ID":"a1764b75-cf72-4747-b57f-04f9a4b75e88","Type":"ContainerStarted","Data":"065408af969d1361013db58e7d7ef96b3a5fe19d9cd10e709c12d8f6c13bea48"} Jun 02 04:47:10.488797 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.488766 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:10.584114 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.584075 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-bundle\") pod \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " Jun 02 04:47:10.584294 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.584157 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-util\") pod \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " Jun 02 04:47:10.584294 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.584250 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwxjp\" (UniqueName: \"kubernetes.io/projected/d8d4ccde-ce9b-4df8-8330-6df14b68872d-kube-api-access-hwxjp\") pod \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\" (UID: \"d8d4ccde-ce9b-4df8-8330-6df14b68872d\") " Jun 02 04:47:10.585246 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.585200 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-bundle" (OuterVolumeSpecName: "bundle") pod "d8d4ccde-ce9b-4df8-8330-6df14b68872d" (UID: "d8d4ccde-ce9b-4df8-8330-6df14b68872d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:47:10.586565 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.586538 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8d4ccde-ce9b-4df8-8330-6df14b68872d-kube-api-access-hwxjp" (OuterVolumeSpecName: "kube-api-access-hwxjp") pod "d8d4ccde-ce9b-4df8-8330-6df14b68872d" (UID: "d8d4ccde-ce9b-4df8-8330-6df14b68872d"). InnerVolumeSpecName "kube-api-access-hwxjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:47:10.685442 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.685363 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwxjp\" (UniqueName: \"kubernetes.io/projected/d8d4ccde-ce9b-4df8-8330-6df14b68872d-kube-api-access-hwxjp\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:47:10.685442 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.685399 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:47:10.702768 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.702729 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-util" (OuterVolumeSpecName: "util") pod "d8d4ccde-ce9b-4df8-8330-6df14b68872d" (UID: "d8d4ccde-ce9b-4df8-8330-6df14b68872d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:47:10.786843 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:10.786807 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d8d4ccde-ce9b-4df8-8330-6df14b68872d-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:47:11.377848 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:11.377805 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" event={"ID":"d8d4ccde-ce9b-4df8-8330-6df14b68872d","Type":"ContainerDied","Data":"e32d707c008312403c57b38acc7a19eabf091d2dac503f8c965f8569dd31eff9"} Jun 02 04:47:11.377848 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:11.377842 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e32d707c008312403c57b38acc7a19eabf091d2dac503f8c965f8569dd31eff9" Jun 02 04:47:11.378345 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:11.377886 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835smt4m" Jun 02 04:47:12.383897 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:12.383864 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" event={"ID":"a1764b75-cf72-4747-b57f-04f9a4b75e88","Type":"ContainerStarted","Data":"6af0f93f8a05f3eefd6947bc9689801a6db33ba978de26ed50c60c17676630c4"} Jun 02 04:47:12.384334 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:12.383917 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:12.408738 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:12.408682 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" podStartSLOduration=1.730069296 podStartE2EDuration="3.408667525s" podCreationTimestamp="2026-06-02 04:47:09 +0000 UTC" firstStartedPulling="2026-06-02 04:47:09.891062016 +0000 UTC m=+312.314832257" lastFinishedPulling="2026-06-02 04:47:11.569660237 +0000 UTC m=+313.993430486" observedRunningTime="2026-06-02 04:47:12.408041654 +0000 UTC m=+314.831811942" watchObservedRunningTime="2026-06-02 04:47:12.408667525 +0000 UTC m=+314.832437788" Jun 02 04:47:17.361321 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:17.361286 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-sjb96" Jun 02 04:47:19.370293 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.370255 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-89j4n" Jun 02 04:47:19.876245 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876204 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk"] Jun 02 04:47:19.876545 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876533 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="extract" Jun 02 04:47:19.876592 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876547 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="extract" Jun 02 04:47:19.876592 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876560 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="util" Jun 02 04:47:19.876592 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876565 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="util" Jun 02 04:47:19.876592 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876574 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="pull" Jun 02 04:47:19.876592 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876580 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="pull" Jun 02 04:47:19.876736 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.876635 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="d8d4ccde-ce9b-4df8-8330-6df14b68872d" containerName="extract" Jun 02 04:47:19.971652 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.971612 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk"] Jun 02 04:47:19.971822 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.971781 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:19.976228 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.976199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:19.976361 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.976258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:19.976422 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.976369 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75gl7\" (UniqueName: \"kubernetes.io/projected/3f41c80b-d73a-4742-9b94-07f060516e9b-kube-api-access-75gl7\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:19.977499 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.977477 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7hkf\"" Jun 02 04:47:19.977614 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.977497 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 04:47:19.979092 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:19.979075 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 04:47:20.077397 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.077354 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.077586 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.077412 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.077586 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.077517 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75gl7\" (UniqueName: \"kubernetes.io/projected/3f41c80b-d73a-4742-9b94-07f060516e9b-kube-api-access-75gl7\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.077846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.077825 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.077886 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.077838 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.091393 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.091361 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75gl7\" (UniqueName: \"kubernetes.io/projected/3f41c80b-d73a-4742-9b94-07f060516e9b-kube-api-access-75gl7\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.281424 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.281334 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:20.423933 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:20.423903 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk"] Jun 02 04:47:20.426989 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:47:20.426958 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f41c80b_d73a_4742_9b94_07f060516e9b.slice/crio-bafefcf0e714e985f4176d9fdb487291d62b78f1511cbd2401054715b82452bc WatchSource:0}: Error finding container bafefcf0e714e985f4176d9fdb487291d62b78f1511cbd2401054715b82452bc: Status 404 returned error can't find the container with id bafefcf0e714e985f4176d9fdb487291d62b78f1511cbd2401054715b82452bc Jun 02 04:47:21.417710 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:21.417618 2566 generic.go:358] "Generic (PLEG): container finished" podID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerID="22aae582f9a7451d204f681a79403da9337d29825b6cfa3099ab2f469a49ea5c" exitCode=0 Jun 02 04:47:21.417861 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:21.417705 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" event={"ID":"3f41c80b-d73a-4742-9b94-07f060516e9b","Type":"ContainerDied","Data":"22aae582f9a7451d204f681a79403da9337d29825b6cfa3099ab2f469a49ea5c"} Jun 02 04:47:21.417861 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:21.417748 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" event={"ID":"3f41c80b-d73a-4742-9b94-07f060516e9b","Type":"ContainerStarted","Data":"bafefcf0e714e985f4176d9fdb487291d62b78f1511cbd2401054715b82452bc"} Jun 02 04:47:23.389353 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:23.389314 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f8df7c74b-c9ngb" Jun 02 04:47:27.442401 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:27.442305 2566 generic.go:358] "Generic (PLEG): container finished" podID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerID="3fd9bdc85c5eca8f67312064059195c86594a4d3d070e76a1f0021c933e3fcf7" exitCode=0 Jun 02 04:47:27.442767 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:27.442400 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" event={"ID":"3f41c80b-d73a-4742-9b94-07f060516e9b","Type":"ContainerDied","Data":"3fd9bdc85c5eca8f67312064059195c86594a4d3d070e76a1f0021c933e3fcf7"} Jun 02 04:47:28.447716 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:28.447680 2566 generic.go:358] "Generic (PLEG): container finished" podID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerID="07f9a8138630a3ee6b86a077a8192f3aff6c572b154c456d2adfea09369dcd91" exitCode=0 Jun 02 04:47:28.448241 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:28.447739 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" event={"ID":"3f41c80b-d73a-4742-9b94-07f060516e9b","Type":"ContainerDied","Data":"07f9a8138630a3ee6b86a077a8192f3aff6c572b154c456d2adfea09369dcd91"} Jun 02 04:47:29.576975 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.576950 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:47:29.657583 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.657546 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-bundle\") pod \"3f41c80b-d73a-4742-9b94-07f060516e9b\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " Jun 02 04:47:29.657757 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.657607 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75gl7\" (UniqueName: \"kubernetes.io/projected/3f41c80b-d73a-4742-9b94-07f060516e9b-kube-api-access-75gl7\") pod \"3f41c80b-d73a-4742-9b94-07f060516e9b\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " Jun 02 04:47:29.657757 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.657637 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-util\") pod \"3f41c80b-d73a-4742-9b94-07f060516e9b\" (UID: \"3f41c80b-d73a-4742-9b94-07f060516e9b\") " Jun 02 04:47:29.658576 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.658546 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-bundle" (OuterVolumeSpecName: "bundle") pod "3f41c80b-d73a-4742-9b94-07f060516e9b" (UID: "3f41c80b-d73a-4742-9b94-07f060516e9b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:47:29.659703 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.659679 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f41c80b-d73a-4742-9b94-07f060516e9b-kube-api-access-75gl7" (OuterVolumeSpecName: "kube-api-access-75gl7") pod "3f41c80b-d73a-4742-9b94-07f060516e9b" (UID: "3f41c80b-d73a-4742-9b94-07f060516e9b"). InnerVolumeSpecName "kube-api-access-75gl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:47:29.662154 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.662089 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-util" (OuterVolumeSpecName: "util") pod "3f41c80b-d73a-4742-9b94-07f060516e9b" (UID: "3f41c80b-d73a-4742-9b94-07f060516e9b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:47:29.758478 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.758387 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:47:29.758478 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.758419 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f41c80b-d73a-4742-9b94-07f060516e9b-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:47:29.758478 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:29.758429 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75gl7\" (UniqueName: \"kubernetes.io/projected/3f41c80b-d73a-4742-9b94-07f060516e9b-kube-api-access-75gl7\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:47:30.456277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:30.456187 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" event={"ID":"3f41c80b-d73a-4742-9b94-07f060516e9b","Type":"ContainerDied","Data":"bafefcf0e714e985f4176d9fdb487291d62b78f1511cbd2401054715b82452bc"} Jun 02 04:47:30.456277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:30.456224 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bafefcf0e714e985f4176d9fdb487291d62b78f1511cbd2401054715b82452bc" Jun 02 04:47:30.456277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:47:30.456227 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avqghk" Jun 02 04:48:18.923773 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.923736 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47"] Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924104 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="extract" Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924118 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="extract" Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924153 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="pull" Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924160 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="pull" Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924167 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="util" Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924172 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="util" Jun 02 04:48:18.924276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.924222 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="3f41c80b-d73a-4742-9b94-07f060516e9b" containerName="extract" Jun 02 04:48:18.927214 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.927196 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:18.929725 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.929703 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 04:48:18.930456 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.930439 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-264pp\"" Jun 02 04:48:18.930510 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.930452 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 04:48:18.936625 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.936598 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47"] Jun 02 04:48:18.981026 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.980980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:18.981243 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.981057 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqc8h\" (UniqueName: \"kubernetes.io/projected/34c85b26-2bb3-4e9c-83a4-d73410512f41-kube-api-access-fqc8h\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:18.981243 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:18.981093 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.081725 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.081678 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fqc8h\" (UniqueName: \"kubernetes.io/projected/34c85b26-2bb3-4e9c-83a4-d73410512f41-kube-api-access-fqc8h\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.081937 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.081747 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.081937 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.081815 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.082213 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.082186 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.082304 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.082233 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.090606 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.090579 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqc8h\" (UniqueName: \"kubernetes.io/projected/34c85b26-2bb3-4e9c-83a4-d73410512f41-kube-api-access-fqc8h\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.237358 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.237263 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:19.370063 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.370038 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47"] Jun 02 04:48:19.372083 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:48:19.372051 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c85b26_2bb3_4e9c_83a4_d73410512f41.slice/crio-324cd5d78cb411a030690f58484bb5a783b362f8d4173f376aabd4751b7dd5e0 WatchSource:0}: Error finding container 324cd5d78cb411a030690f58484bb5a783b362f8d4173f376aabd4751b7dd5e0: Status 404 returned error can't find the container with id 324cd5d78cb411a030690f58484bb5a783b362f8d4173f376aabd4751b7dd5e0 Jun 02 04:48:19.519390 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.519297 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6"] Jun 02 04:48:19.522844 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.522819 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.531084 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.531057 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6"] Jun 02 04:48:19.586489 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.586446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.586669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.586560 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.586669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.586611 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj56m\" (UniqueName: \"kubernetes.io/projected/20c8f841-3813-43cd-adc2-00d2a79dacb4-kube-api-access-mj56m\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.633272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.633230 2566 generic.go:358] "Generic (PLEG): container finished" podID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerID="c463127da962e7d6a398d4d773c6cc59729633328bd5ee110bab7ba20b3d157f" exitCode=0 Jun 02 04:48:19.633430 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.633311 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" event={"ID":"34c85b26-2bb3-4e9c-83a4-d73410512f41","Type":"ContainerDied","Data":"c463127da962e7d6a398d4d773c6cc59729633328bd5ee110bab7ba20b3d157f"} Jun 02 04:48:19.633430 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.633336 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" event={"ID":"34c85b26-2bb3-4e9c-83a4-d73410512f41","Type":"ContainerStarted","Data":"324cd5d78cb411a030690f58484bb5a783b362f8d4173f376aabd4751b7dd5e0"} Jun 02 04:48:19.687217 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.687175 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.687424 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.687227 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mj56m\" (UniqueName: \"kubernetes.io/projected/20c8f841-3813-43cd-adc2-00d2a79dacb4-kube-api-access-mj56m\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.687424 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.687274 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.687652 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.687629 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.687721 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.687664 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.695994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.695970 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj56m\" (UniqueName: \"kubernetes.io/projected/20c8f841-3813-43cd-adc2-00d2a79dacb4-kube-api-access-mj56m\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.848009 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.847971 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:19.923204 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.923173 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz"] Jun 02 04:48:19.928048 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.928020 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:19.935154 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.935113 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz"] Jun 02 04:48:19.979739 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.979711 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6"] Jun 02 04:48:19.981620 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:48:19.981586 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c8f841_3813_43cd_adc2_00d2a79dacb4.slice/crio-c6be44b727062c67271798dca5dfac0df4d1bffeab0d7c536acbce8a40fe7869 WatchSource:0}: Error finding container c6be44b727062c67271798dca5dfac0df4d1bffeab0d7c536acbce8a40fe7869: Status 404 returned error can't find the container with id c6be44b727062c67271798dca5dfac0df4d1bffeab0d7c536acbce8a40fe7869 Jun 02 04:48:19.990506 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.990477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:19.990617 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.990515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:19.990617 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:19.990540 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psgt7\" (UniqueName: \"kubernetes.io/projected/cb362b3b-24d4-415c-a209-48a9884ec986-kube-api-access-psgt7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.091527 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.091497 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.091670 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.091538 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.091670 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.091657 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-psgt7\" (UniqueName: \"kubernetes.io/projected/cb362b3b-24d4-415c-a209-48a9884ec986-kube-api-access-psgt7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.091869 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.091851 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.091956 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.091922 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.101681 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.101625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psgt7\" (UniqueName: \"kubernetes.io/projected/cb362b3b-24d4-415c-a209-48a9884ec986-kube-api-access-psgt7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.240657 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.240619 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:20.327834 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.327804 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8"] Jun 02 04:48:20.331581 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.331561 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.340941 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.340910 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8"] Jun 02 04:48:20.394865 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.394836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pngqq\" (UniqueName: \"kubernetes.io/projected/0082c667-16fd-4340-b96b-18e6b73b4f7f-kube-api-access-pngqq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.394987 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.394953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.394987 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.394982 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.405181 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.405152 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz"] Jun 02 04:48:20.407105 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:48:20.407078 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb362b3b_24d4_415c_a209_48a9884ec986.slice/crio-4d6967a6d13876f636a2d7bd2be827782800662c37c1c4ef271d1adc00879a5c WatchSource:0}: Error finding container 4d6967a6d13876f636a2d7bd2be827782800662c37c1c4ef271d1adc00879a5c: Status 404 returned error can't find the container with id 4d6967a6d13876f636a2d7bd2be827782800662c37c1c4ef271d1adc00879a5c Jun 02 04:48:20.495819 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.495791 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.495958 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.495828 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.495958 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.495863 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pngqq\" (UniqueName: \"kubernetes.io/projected/0082c667-16fd-4340-b96b-18e6b73b4f7f-kube-api-access-pngqq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.496219 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.496198 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.496264 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.496231 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.505290 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.505262 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pngqq\" (UniqueName: \"kubernetes.io/projected/0082c667-16fd-4340-b96b-18e6b73b4f7f-kube-api-access-pngqq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.638647 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.638555 2566 generic.go:358] "Generic (PLEG): container finished" podID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerID="6b9e0d5dd9414f3473dae7dbe0393e5c6f43eb8c17b118e7252c38a1feeece4e" exitCode=0 Jun 02 04:48:20.638806 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.638648 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" event={"ID":"20c8f841-3813-43cd-adc2-00d2a79dacb4","Type":"ContainerDied","Data":"6b9e0d5dd9414f3473dae7dbe0393e5c6f43eb8c17b118e7252c38a1feeece4e"} Jun 02 04:48:20.638806 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.638687 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" event={"ID":"20c8f841-3813-43cd-adc2-00d2a79dacb4","Type":"ContainerStarted","Data":"c6be44b727062c67271798dca5dfac0df4d1bffeab0d7c536acbce8a40fe7869"} Jun 02 04:48:20.640038 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.640015 2566 generic.go:358] "Generic (PLEG): container finished" podID="cb362b3b-24d4-415c-a209-48a9884ec986" containerID="173a21f408d01268be43aa6be2a3e257ed66747fb6aeba5d2813ebfb45c4a390" exitCode=0 Jun 02 04:48:20.640159 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.640099 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" event={"ID":"cb362b3b-24d4-415c-a209-48a9884ec986","Type":"ContainerDied","Data":"173a21f408d01268be43aa6be2a3e257ed66747fb6aeba5d2813ebfb45c4a390"} Jun 02 04:48:20.640159 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.640127 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" event={"ID":"cb362b3b-24d4-415c-a209-48a9884ec986","Type":"ContainerStarted","Data":"4d6967a6d13876f636a2d7bd2be827782800662c37c1c4ef271d1adc00879a5c"} Jun 02 04:48:20.641994 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.641971 2566 generic.go:358] "Generic (PLEG): container finished" podID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerID="eafa329016562b74fa9075ac5ba53183321947aa31cff9b478aa4ba42ce544ba" exitCode=0 Jun 02 04:48:20.642083 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.642004 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" event={"ID":"34c85b26-2bb3-4e9c-83a4-d73410512f41","Type":"ContainerDied","Data":"eafa329016562b74fa9075ac5ba53183321947aa31cff9b478aa4ba42ce544ba"} Jun 02 04:48:20.657118 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.657086 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:20.802334 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:20.802309 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8"] Jun 02 04:48:20.826046 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:48:20.826012 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0082c667_16fd_4340_b96b_18e6b73b4f7f.slice/crio-9c2a508a6a675826f35b2cd7adb8385390aa1b59c963826e5fd22f047c896193 WatchSource:0}: Error finding container 9c2a508a6a675826f35b2cd7adb8385390aa1b59c963826e5fd22f047c896193: Status 404 returned error can't find the container with id 9c2a508a6a675826f35b2cd7adb8385390aa1b59c963826e5fd22f047c896193 Jun 02 04:48:21.648600 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.648563 2566 generic.go:358] "Generic (PLEG): container finished" podID="cb362b3b-24d4-415c-a209-48a9884ec986" containerID="cf5d8641d3cf916df0b713043cbc99e93cb5c087ec1bec6027b2455c7126ee99" exitCode=0 Jun 02 04:48:21.649036 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.648638 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" event={"ID":"cb362b3b-24d4-415c-a209-48a9884ec986","Type":"ContainerDied","Data":"cf5d8641d3cf916df0b713043cbc99e93cb5c087ec1bec6027b2455c7126ee99"} Jun 02 04:48:21.650645 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.650616 2566 generic.go:358] "Generic (PLEG): container finished" podID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerID="e8d68442c46c535be8f8399d8f6a91076f50d0a5fbb2ce1eee36e97976775b9d" exitCode=0 Jun 02 04:48:21.650739 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.650715 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" event={"ID":"34c85b26-2bb3-4e9c-83a4-d73410512f41","Type":"ContainerDied","Data":"e8d68442c46c535be8f8399d8f6a91076f50d0a5fbb2ce1eee36e97976775b9d"} Jun 02 04:48:21.652264 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.652240 2566 generic.go:358] "Generic (PLEG): container finished" podID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerID="52ca33da20ed1ed7e5498e8b0923ef9749b820e0703ad52aa1b663f711558ee9" exitCode=0 Jun 02 04:48:21.652354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.652310 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" event={"ID":"0082c667-16fd-4340-b96b-18e6b73b4f7f","Type":"ContainerDied","Data":"52ca33da20ed1ed7e5498e8b0923ef9749b820e0703ad52aa1b663f711558ee9"} Jun 02 04:48:21.652354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.652338 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" event={"ID":"0082c667-16fd-4340-b96b-18e6b73b4f7f","Type":"ContainerStarted","Data":"9c2a508a6a675826f35b2cd7adb8385390aa1b59c963826e5fd22f047c896193"} Jun 02 04:48:21.654233 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:21.654212 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" event={"ID":"20c8f841-3813-43cd-adc2-00d2a79dacb4","Type":"ContainerStarted","Data":"a422403f8661dfbdd542b9f16dd8d7c514cb41d8c0792eb71f1a13e99a4bf696"} Jun 02 04:48:22.659737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.659703 2566 generic.go:358] "Generic (PLEG): container finished" podID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerID="a422403f8661dfbdd542b9f16dd8d7c514cb41d8c0792eb71f1a13e99a4bf696" exitCode=0 Jun 02 04:48:22.659737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.659733 2566 generic.go:358] "Generic (PLEG): container finished" podID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerID="209629aca45765c76197176edf985c13fbfa2e6226c54b0309a77a9f1246af5c" exitCode=0 Jun 02 04:48:22.660224 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.659792 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" event={"ID":"20c8f841-3813-43cd-adc2-00d2a79dacb4","Type":"ContainerDied","Data":"a422403f8661dfbdd542b9f16dd8d7c514cb41d8c0792eb71f1a13e99a4bf696"} Jun 02 04:48:22.660224 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.659826 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" event={"ID":"20c8f841-3813-43cd-adc2-00d2a79dacb4","Type":"ContainerDied","Data":"209629aca45765c76197176edf985c13fbfa2e6226c54b0309a77a9f1246af5c"} Jun 02 04:48:22.661591 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.661561 2566 generic.go:358] "Generic (PLEG): container finished" podID="cb362b3b-24d4-415c-a209-48a9884ec986" containerID="714b2e3e8055e2810d1ca9ba853b57c86f5efd96ee43040c873bc4c90869ffe7" exitCode=0 Jun 02 04:48:22.661725 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.661646 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" event={"ID":"cb362b3b-24d4-415c-a209-48a9884ec986","Type":"ContainerDied","Data":"714b2e3e8055e2810d1ca9ba853b57c86f5efd96ee43040c873bc4c90869ffe7"} Jun 02 04:48:22.663271 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.663249 2566 generic.go:358] "Generic (PLEG): container finished" podID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerID="ed5b1a7dbd216f86bb88cdb231e06ad23ef2521e7c1096d92ce99a27fff10005" exitCode=0 Jun 02 04:48:22.663372 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.663332 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" event={"ID":"0082c667-16fd-4340-b96b-18e6b73b4f7f","Type":"ContainerDied","Data":"ed5b1a7dbd216f86bb88cdb231e06ad23ef2521e7c1096d92ce99a27fff10005"} Jun 02 04:48:22.792982 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.792954 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:22.815480 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.815431 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-util\") pod \"34c85b26-2bb3-4e9c-83a4-d73410512f41\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " Jun 02 04:48:22.815480 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.815475 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqc8h\" (UniqueName: \"kubernetes.io/projected/34c85b26-2bb3-4e9c-83a4-d73410512f41-kube-api-access-fqc8h\") pod \"34c85b26-2bb3-4e9c-83a4-d73410512f41\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " Jun 02 04:48:22.815733 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.815524 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-bundle\") pod \"34c85b26-2bb3-4e9c-83a4-d73410512f41\" (UID: \"34c85b26-2bb3-4e9c-83a4-d73410512f41\") " Jun 02 04:48:22.816119 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.816081 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-bundle" (OuterVolumeSpecName: "bundle") pod "34c85b26-2bb3-4e9c-83a4-d73410512f41" (UID: "34c85b26-2bb3-4e9c-83a4-d73410512f41"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:22.817667 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.817640 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c85b26-2bb3-4e9c-83a4-d73410512f41-kube-api-access-fqc8h" (OuterVolumeSpecName: "kube-api-access-fqc8h") pod "34c85b26-2bb3-4e9c-83a4-d73410512f41" (UID: "34c85b26-2bb3-4e9c-83a4-d73410512f41"). InnerVolumeSpecName "kube-api-access-fqc8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:48:22.821884 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.821858 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-util" (OuterVolumeSpecName: "util") pod "34c85b26-2bb3-4e9c-83a4-d73410512f41" (UID: "34c85b26-2bb3-4e9c-83a4-d73410512f41"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:22.916332 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.916295 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:22.916332 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.916326 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqc8h\" (UniqueName: \"kubernetes.io/projected/34c85b26-2bb3-4e9c-83a4-d73410512f41-kube-api-access-fqc8h\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:22.916332 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:22.916339 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34c85b26-2bb3-4e9c-83a4-d73410512f41-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:23.668457 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.668425 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" Jun 02 04:48:23.668881 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.668422 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47" event={"ID":"34c85b26-2bb3-4e9c-83a4-d73410512f41","Type":"ContainerDied","Data":"324cd5d78cb411a030690f58484bb5a783b362f8d4173f376aabd4751b7dd5e0"} Jun 02 04:48:23.668881 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.668546 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="324cd5d78cb411a030690f58484bb5a783b362f8d4173f376aabd4751b7dd5e0" Jun 02 04:48:23.670215 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.670188 2566 generic.go:358] "Generic (PLEG): container finished" podID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerID="d6786f5c9ab485478473c4ea8917efcd4f4c8e2b9bf5734f922f98aa745c29dd" exitCode=0 Jun 02 04:48:23.670317 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.670260 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" event={"ID":"0082c667-16fd-4340-b96b-18e6b73b4f7f","Type":"ContainerDied","Data":"d6786f5c9ab485478473c4ea8917efcd4f4c8e2b9bf5734f922f98aa745c29dd"} Jun 02 04:48:23.821171 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.821127 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:23.824338 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.824319 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:23.924465 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924363 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-bundle\") pod \"20c8f841-3813-43cd-adc2-00d2a79dacb4\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " Jun 02 04:48:23.924465 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924428 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-bundle\") pod \"cb362b3b-24d4-415c-a209-48a9884ec986\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " Jun 02 04:48:23.924465 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924455 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-util\") pod \"20c8f841-3813-43cd-adc2-00d2a79dacb4\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " Jun 02 04:48:23.924734 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924498 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psgt7\" (UniqueName: \"kubernetes.io/projected/cb362b3b-24d4-415c-a209-48a9884ec986-kube-api-access-psgt7\") pod \"cb362b3b-24d4-415c-a209-48a9884ec986\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " Jun 02 04:48:23.924734 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924573 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj56m\" (UniqueName: \"kubernetes.io/projected/20c8f841-3813-43cd-adc2-00d2a79dacb4-kube-api-access-mj56m\") pod \"20c8f841-3813-43cd-adc2-00d2a79dacb4\" (UID: \"20c8f841-3813-43cd-adc2-00d2a79dacb4\") " Jun 02 04:48:23.924734 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924597 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-util\") pod \"cb362b3b-24d4-415c-a209-48a9884ec986\" (UID: \"cb362b3b-24d4-415c-a209-48a9884ec986\") " Jun 02 04:48:23.925084 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.924933 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-bundle" (OuterVolumeSpecName: "bundle") pod "20c8f841-3813-43cd-adc2-00d2a79dacb4" (UID: "20c8f841-3813-43cd-adc2-00d2a79dacb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:23.925365 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.925328 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-bundle" (OuterVolumeSpecName: "bundle") pod "cb362b3b-24d4-415c-a209-48a9884ec986" (UID: "cb362b3b-24d4-415c-a209-48a9884ec986"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:23.926868 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.926840 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20c8f841-3813-43cd-adc2-00d2a79dacb4-kube-api-access-mj56m" (OuterVolumeSpecName: "kube-api-access-mj56m") pod "20c8f841-3813-43cd-adc2-00d2a79dacb4" (UID: "20c8f841-3813-43cd-adc2-00d2a79dacb4"). InnerVolumeSpecName "kube-api-access-mj56m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:48:23.927011 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.926993 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb362b3b-24d4-415c-a209-48a9884ec986-kube-api-access-psgt7" (OuterVolumeSpecName: "kube-api-access-psgt7") pod "cb362b3b-24d4-415c-a209-48a9884ec986" (UID: "cb362b3b-24d4-415c-a209-48a9884ec986"). InnerVolumeSpecName "kube-api-access-psgt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:48:23.929824 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.929795 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-util" (OuterVolumeSpecName: "util") pod "cb362b3b-24d4-415c-a209-48a9884ec986" (UID: "cb362b3b-24d4-415c-a209-48a9884ec986"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:23.930981 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:23.930959 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-util" (OuterVolumeSpecName: "util") pod "20c8f841-3813-43cd-adc2-00d2a79dacb4" (UID: "20c8f841-3813-43cd-adc2-00d2a79dacb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:24.025777 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.025737 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj56m\" (UniqueName: \"kubernetes.io/projected/20c8f841-3813-43cd-adc2-00d2a79dacb4-kube-api-access-mj56m\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.025777 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.025771 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.025777 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.025782 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.025777 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.025791 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb362b3b-24d4-415c-a209-48a9884ec986-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.026040 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.025799 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20c8f841-3813-43cd-adc2-00d2a79dacb4-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.026040 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.025807 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psgt7\" (UniqueName: \"kubernetes.io/projected/cb362b3b-24d4-415c-a209-48a9884ec986-kube-api-access-psgt7\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.675152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.675100 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" event={"ID":"20c8f841-3813-43cd-adc2-00d2a79dacb4","Type":"ContainerDied","Data":"c6be44b727062c67271798dca5dfac0df4d1bffeab0d7c536acbce8a40fe7869"} Jun 02 04:48:24.675152 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.675121 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6" Jun 02 04:48:24.675613 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.675167 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6be44b727062c67271798dca5dfac0df4d1bffeab0d7c536acbce8a40fe7869" Jun 02 04:48:24.676846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.676823 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" Jun 02 04:48:24.676947 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.676852 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz" event={"ID":"cb362b3b-24d4-415c-a209-48a9884ec986","Type":"ContainerDied","Data":"4d6967a6d13876f636a2d7bd2be827782800662c37c1c4ef271d1adc00879a5c"} Jun 02 04:48:24.676947 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.676878 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d6967a6d13876f636a2d7bd2be827782800662c37c1c4ef271d1adc00879a5c" Jun 02 04:48:24.805941 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.805916 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:24.833165 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.833118 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-util\") pod \"0082c667-16fd-4340-b96b-18e6b73b4f7f\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " Jun 02 04:48:24.833303 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.833250 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pngqq\" (UniqueName: \"kubernetes.io/projected/0082c667-16fd-4340-b96b-18e6b73b4f7f-kube-api-access-pngqq\") pod \"0082c667-16fd-4340-b96b-18e6b73b4f7f\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " Jun 02 04:48:24.833303 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.833290 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-bundle\") pod \"0082c667-16fd-4340-b96b-18e6b73b4f7f\" (UID: \"0082c667-16fd-4340-b96b-18e6b73b4f7f\") " Jun 02 04:48:24.833910 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.833868 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-bundle" (OuterVolumeSpecName: "bundle") pod "0082c667-16fd-4340-b96b-18e6b73b4f7f" (UID: "0082c667-16fd-4340-b96b-18e6b73b4f7f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:24.835456 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.835425 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0082c667-16fd-4340-b96b-18e6b73b4f7f-kube-api-access-pngqq" (OuterVolumeSpecName: "kube-api-access-pngqq") pod "0082c667-16fd-4340-b96b-18e6b73b4f7f" (UID: "0082c667-16fd-4340-b96b-18e6b73b4f7f"). InnerVolumeSpecName "kube-api-access-pngqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:48:24.838911 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.838884 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-util" (OuterVolumeSpecName: "util") pod "0082c667-16fd-4340-b96b-18e6b73b4f7f" (UID: "0082c667-16fd-4340-b96b-18e6b73b4f7f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:48:24.934502 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.934419 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.934502 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.934450 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0082c667-16fd-4340-b96b-18e6b73b4f7f-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:24.934502 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:24.934459 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pngqq\" (UniqueName: \"kubernetes.io/projected/0082c667-16fd-4340-b96b-18e6b73b4f7f-kube-api-access-pngqq\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:48:25.682256 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:25.682220 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" Jun 02 04:48:25.682647 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:25.682212 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8" event={"ID":"0082c667-16fd-4340-b96b-18e6b73b4f7f","Type":"ContainerDied","Data":"9c2a508a6a675826f35b2cd7adb8385390aa1b59c963826e5fd22f047c896193"} Jun 02 04:48:25.682647 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:25.682341 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c2a508a6a675826f35b2cd7adb8385390aa1b59c963826e5fd22f047c896193" Jun 02 04:48:27.206554 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.206505 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-764cf7446c-gzrzr"] Jun 02 04:48:27.207271 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207250 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="pull" Jun 02 04:48:27.207271 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207273 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="pull" Jun 02 04:48:27.207429 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207299 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="extract" Jun 02 04:48:27.207429 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207309 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="extract" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207442 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="util" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207462 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="util" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207478 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="extract" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207488 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="extract" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207507 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="extract" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207516 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="extract" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207531 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="util" Jun 02 04:48:27.207557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207540 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="util" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207562 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="pull" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207570 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="pull" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207579 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="extract" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207588 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="extract" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207605 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="pull" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207614 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="pull" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207681 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="util" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207691 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="util" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207712 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="pull" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207721 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="pull" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207739 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="util" Jun 02 04:48:27.207918 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207799 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="util" Jun 02 04:48:27.208500 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.207988 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="20c8f841-3813-43cd-adc2-00d2a79dacb4" containerName="extract" Jun 02 04:48:27.208500 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.208005 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="cb362b3b-24d4-415c-a209-48a9884ec986" containerName="extract" Jun 02 04:48:27.208500 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.208018 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="0082c667-16fd-4340-b96b-18e6b73b4f7f" containerName="extract" Jun 02 04:48:27.208500 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.208041 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="34c85b26-2bb3-4e9c-83a4-d73410512f41" containerName="extract" Jun 02 04:48:27.212952 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.212927 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.218415 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.218388 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-764cf7446c-gzrzr"] Jun 02 04:48:27.255324 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255283 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-trusted-ca-bundle\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.255324 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-oauth-config\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.255608 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255404 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4rjf\" (UniqueName: \"kubernetes.io/projected/e1473022-da19-4ce4-aee9-1b46c86a8ded-kube-api-access-s4rjf\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.255608 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255479 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-serving-cert\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.255608 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-oauth-serving-cert\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.255608 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-config\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.255608 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.255589 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-service-ca\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356232 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-oauth-serving-cert\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356277 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356280 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-config\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356310 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-service-ca\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356367 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-trusted-ca-bundle\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356396 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-oauth-config\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356429 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s4rjf\" (UniqueName: \"kubernetes.io/projected/e1473022-da19-4ce4-aee9-1b46c86a8ded-kube-api-access-s4rjf\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.356526 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.356471 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-serving-cert\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.357113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.357080 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-service-ca\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.357247 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.357094 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-oauth-serving-cert\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.357304 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.357272 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-config\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.357382 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.357358 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1473022-da19-4ce4-aee9-1b46c86a8ded-trusted-ca-bundle\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.358979 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.358948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-oauth-config\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.359124 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.359103 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1473022-da19-4ce4-aee9-1b46c86a8ded-console-serving-cert\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.365968 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.365944 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4rjf\" (UniqueName: \"kubernetes.io/projected/e1473022-da19-4ce4-aee9-1b46c86a8ded-kube-api-access-s4rjf\") pod \"console-764cf7446c-gzrzr\" (UID: \"e1473022-da19-4ce4-aee9-1b46c86a8ded\") " pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.524404 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.524312 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:27.658996 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.658964 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-764cf7446c-gzrzr"] Jun 02 04:48:27.660766 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:48:27.660738 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1473022_da19_4ce4_aee9_1b46c86a8ded.slice/crio-17f1c7a3c64fbba09631d8e070fe812baacca0ec38e9466850c4084dabc8334d WatchSource:0}: Error finding container 17f1c7a3c64fbba09631d8e070fe812baacca0ec38e9466850c4084dabc8334d: Status 404 returned error can't find the container with id 17f1c7a3c64fbba09631d8e070fe812baacca0ec38e9466850c4084dabc8334d Jun 02 04:48:27.690928 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:27.690897 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764cf7446c-gzrzr" event={"ID":"e1473022-da19-4ce4-aee9-1b46c86a8ded","Type":"ContainerStarted","Data":"17f1c7a3c64fbba09631d8e070fe812baacca0ec38e9466850c4084dabc8334d"} Jun 02 04:48:28.695547 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:28.695505 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764cf7446c-gzrzr" event={"ID":"e1473022-da19-4ce4-aee9-1b46c86a8ded","Type":"ContainerStarted","Data":"7e21fed7dafaea6b2ff38cfc2bda80cbb687797fb4fbb7618ec959aebec002f9"} Jun 02 04:48:28.713472 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:28.713426 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-764cf7446c-gzrzr" podStartSLOduration=1.713411762 podStartE2EDuration="1.713411762s" podCreationTimestamp="2026-06-02 04:48:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:48:28.712171167 +0000 UTC m=+391.135941430" watchObservedRunningTime="2026-06-02 04:48:28.713411762 +0000 UTC m=+391.137182025" Jun 02 04:48:37.525028 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:37.524982 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:37.525028 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:37.525040 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:37.530495 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:37.530469 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:37.731754 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:37.731720 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-764cf7446c-gzrzr" Jun 02 04:48:37.797557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:37.797478 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d6dd5966d-bsqll"] Jun 02 04:48:55.229766 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.229727 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd"] Jun 02 04:48:55.234661 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.234638 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.238529 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.238502 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 04:48:55.238529 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.238506 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 04:48:55.238697 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.238530 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 04:48:55.239473 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.239455 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-264pp\"" Jun 02 04:48:55.239536 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.239494 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 04:48:55.247176 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.247153 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd"] Jun 02 04:48:55.409894 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.409855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37ee2087-7e53-43f6-b277-6ff3328240a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.409894 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.409898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjctp\" (UniqueName: \"kubernetes.io/projected/37ee2087-7e53-43f6-b277-6ff3328240a4-kube-api-access-gjctp\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.410119 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.409928 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37ee2087-7e53-43f6-b277-6ff3328240a4-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.510998 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.510901 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37ee2087-7e53-43f6-b277-6ff3328240a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.510998 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.510948 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjctp\" (UniqueName: \"kubernetes.io/projected/37ee2087-7e53-43f6-b277-6ff3328240a4-kube-api-access-gjctp\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.510998 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.510984 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37ee2087-7e53-43f6-b277-6ff3328240a4-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.511643 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.511617 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37ee2087-7e53-43f6-b277-6ff3328240a4-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.513330 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.513308 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37ee2087-7e53-43f6-b277-6ff3328240a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.520388 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.520362 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjctp\" (UniqueName: \"kubernetes.io/projected/37ee2087-7e53-43f6-b277-6ff3328240a4-kube-api-access-gjctp\") pod \"kuadrant-console-plugin-6cb54b5c86-z4hdd\" (UID: \"37ee2087-7e53-43f6-b277-6ff3328240a4\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.545188 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.545127 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" Jun 02 04:48:55.699765 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.699737 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd"] Jun 02 04:48:55.701652 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:48:55.701622 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37ee2087_7e53_43f6_b277_6ff3328240a4.slice/crio-b79df1001133e0b57a619ac49f8e69fafc21262ecfbb6c8476ba8f4a3bb21fa8 WatchSource:0}: Error finding container b79df1001133e0b57a619ac49f8e69fafc21262ecfbb6c8476ba8f4a3bb21fa8: Status 404 returned error can't find the container with id b79df1001133e0b57a619ac49f8e69fafc21262ecfbb6c8476ba8f4a3bb21fa8 Jun 02 04:48:55.793866 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:48:55.793776 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" event={"ID":"37ee2087-7e53-43f6-b277-6ff3328240a4","Type":"ContainerStarted","Data":"b79df1001133e0b57a619ac49f8e69fafc21262ecfbb6c8476ba8f4a3bb21fa8"} Jun 02 04:49:02.823038 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:02.822969 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6d6dd5966d-bsqll" podUID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" containerName="console" containerID="cri-o://6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097" gracePeriod=15 Jun 02 04:49:02.909314 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:02.909274 2566 patch_prober.go:28] interesting pod/console-6d6dd5966d-bsqll container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body= Jun 02 04:49:02.909502 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:02.909340 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-6d6dd5966d-bsqll" podUID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" Jun 02 04:49:03.090150 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.090104 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d6dd5966d-bsqll_4c2ac6b4-6f68-40d4-a526-89de8e2ee227/console/0.log" Jun 02 04:49:03.090288 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.090200 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:49:03.184649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.184614 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmgt5\" (UniqueName: \"kubernetes.io/projected/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-kube-api-access-zmgt5\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.184649 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.184655 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-service-ca\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.184974 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.184693 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-trusted-ca-bundle\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.184974 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.184724 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-serving-cert\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.184974 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.184931 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-config\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.185178 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185016 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-oauth-config\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.185178 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185063 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-oauth-serving-cert\") pod \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\" (UID: \"4c2ac6b4-6f68-40d4-a526-89de8e2ee227\") " Jun 02 04:49:03.185295 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185239 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:49:03.185407 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185387 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.185467 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185429 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:49:03.185586 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185543 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:49:03.185709 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.185605 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-config" (OuterVolumeSpecName: "console-config") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:49:03.187595 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.187551 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-kube-api-access-zmgt5" (OuterVolumeSpecName: "kube-api-access-zmgt5") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "kube-api-access-zmgt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:49:03.187722 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.187697 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:49:03.188043 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.188013 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c2ac6b4-6f68-40d4-a526-89de8e2ee227" (UID: "4c2ac6b4-6f68-40d4-a526-89de8e2ee227"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:49:03.286542 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.286501 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmgt5\" (UniqueName: \"kubernetes.io/projected/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-kube-api-access-zmgt5\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.286542 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.286540 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.286730 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.286554 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.286730 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.286564 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.286730 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.286573 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-console-oauth-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.286730 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.286581 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2ac6b4-6f68-40d4-a526-89de8e2ee227-oauth-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:49:03.829440 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.829409 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d6dd5966d-bsqll_4c2ac6b4-6f68-40d4-a526-89de8e2ee227/console/0.log" Jun 02 04:49:03.829907 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.829458 2566 generic.go:358] "Generic (PLEG): container finished" podID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" containerID="6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097" exitCode=2 Jun 02 04:49:03.829907 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.829546 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dd5966d-bsqll" event={"ID":"4c2ac6b4-6f68-40d4-a526-89de8e2ee227","Type":"ContainerDied","Data":"6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097"} Jun 02 04:49:03.829907 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.829565 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6dd5966d-bsqll" Jun 02 04:49:03.829907 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.829592 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6dd5966d-bsqll" event={"ID":"4c2ac6b4-6f68-40d4-a526-89de8e2ee227","Type":"ContainerDied","Data":"db9f189b211f19d5a5e613399780165940d5103f93338be927d4e5c591d76829"} Jun 02 04:49:03.829907 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.829613 2566 scope.go:117] "RemoveContainer" containerID="6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097" Jun 02 04:49:03.884203 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.884168 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d6dd5966d-bsqll"] Jun 02 04:49:03.892889 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:03.892853 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d6dd5966d-bsqll"] Jun 02 04:49:04.121479 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:04.121390 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" path="/var/lib/kubelet/pods/4c2ac6b4-6f68-40d4-a526-89de8e2ee227/volumes" Jun 02 04:49:17.975315 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:17.975118 2566 scope.go:117] "RemoveContainer" containerID="6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097" Jun 02 04:49:17.975581 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:49:17.975503 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097\": container with ID starting with 6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097 not found: ID does not exist" containerID="6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097" Jun 02 04:49:17.975581 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:17.975528 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097"} err="failed to get container status \"6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097\": rpc error: code = NotFound desc = could not find container \"6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097\": container with ID starting with 6d163e94837a521461136583767770c6af696feaf8c98040dfddedc1da942097 not found: ID does not exist" Jun 02 04:49:18.891052 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:18.891011 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" event={"ID":"37ee2087-7e53-43f6-b277-6ff3328240a4","Type":"ContainerStarted","Data":"b1e2a1e3eb5ec644ca10f75573881a5d34633c26b6c8c15a205837b2e575e9d1"} Jun 02 04:49:18.908682 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:18.908633 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-z4hdd" podStartSLOduration=1.557385437 podStartE2EDuration="23.908618685s" podCreationTimestamp="2026-06-02 04:48:55 +0000 UTC" firstStartedPulling="2026-06-02 04:48:55.70284879 +0000 UTC m=+418.126619034" lastFinishedPulling="2026-06-02 04:49:18.05408203 +0000 UTC m=+440.477852282" observedRunningTime="2026-06-02 04:49:18.907134355 +0000 UTC m=+441.330904621" watchObservedRunningTime="2026-06-02 04:49:18.908618685 +0000 UTC m=+441.332388948" Jun 02 04:49:51.947926 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.947885 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:49:51.948371 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.948237 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" containerName="console" Jun 02 04:49:51.948371 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.948249 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" containerName="console" Jun 02 04:49:51.948371 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.948315 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="4c2ac6b4-6f68-40d4-a526-89de8e2ee227" containerName="console" Jun 02 04:49:51.982636 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.982600 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:49:51.982803 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.982720 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:51.985234 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:51.985208 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 04:49:52.032797 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.032761 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d9436d4-9319-4945-bcb7-b476d9f8e775-config-file\") pod \"limitador-limitador-7d549b5b-xb2qr\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.032956 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.032861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7gbq\" (UniqueName: \"kubernetes.io/projected/9d9436d4-9319-4945-bcb7-b476d9f8e775-kube-api-access-k7gbq\") pod \"limitador-limitador-7d549b5b-xb2qr\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.046826 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.046791 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:49:52.134300 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.134262 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d9436d4-9319-4945-bcb7-b476d9f8e775-config-file\") pod \"limitador-limitador-7d549b5b-xb2qr\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.134504 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.134356 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k7gbq\" (UniqueName: \"kubernetes.io/projected/9d9436d4-9319-4945-bcb7-b476d9f8e775-kube-api-access-k7gbq\") pod \"limitador-limitador-7d549b5b-xb2qr\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.134996 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.134972 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d9436d4-9319-4945-bcb7-b476d9f8e775-config-file\") pod \"limitador-limitador-7d549b5b-xb2qr\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.144359 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.144335 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7gbq\" (UniqueName: \"kubernetes.io/projected/9d9436d4-9319-4945-bcb7-b476d9f8e775-kube-api-access-k7gbq\") pod \"limitador-limitador-7d549b5b-xb2qr\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.292492 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.292393 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:52.426691 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.426664 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:49:52.429121 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:49:52.429094 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9436d4_9319_4945_bcb7_b476d9f8e775.slice/crio-96ea499d0bdab68ece6b02abe3a7e64aadbdb01935f960d448334ce10a69c3c9 WatchSource:0}: Error finding container 96ea499d0bdab68ece6b02abe3a7e64aadbdb01935f960d448334ce10a69c3c9: Status 404 returned error can't find the container with id 96ea499d0bdab68ece6b02abe3a7e64aadbdb01935f960d448334ce10a69c3c9 Jun 02 04:49:52.771219 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.771169 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mg9xz"] Jun 02 04:49:52.775716 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.775518 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:49:52.777897 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.777877 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jrnl9\"" Jun 02 04:49:52.789471 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.789446 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mg9xz"] Jun 02 04:49:52.840230 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.840192 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4zjg\" (UniqueName: \"kubernetes.io/projected/024a291f-8937-402c-86b5-3b3bc991a77e-kube-api-access-k4zjg\") pod \"authorino-f99f4b5cd-mg9xz\" (UID: \"024a291f-8937-402c-86b5-3b3bc991a77e\") " pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:49:52.938949 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.938915 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-9kx9c"] Jun 02 04:49:52.940921 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.940897 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k4zjg\" (UniqueName: \"kubernetes.io/projected/024a291f-8937-402c-86b5-3b3bc991a77e-kube-api-access-k4zjg\") pod \"authorino-f99f4b5cd-mg9xz\" (UID: \"024a291f-8937-402c-86b5-3b3bc991a77e\") " pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:49:52.942243 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.942226 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:49:52.947267 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.947239 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-9kx9c"] Jun 02 04:49:52.954006 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:52.953977 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4zjg\" (UniqueName: \"kubernetes.io/projected/024a291f-8937-402c-86b5-3b3bc991a77e-kube-api-access-k4zjg\") pod \"authorino-f99f4b5cd-mg9xz\" (UID: \"024a291f-8937-402c-86b5-3b3bc991a77e\") " pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:49:53.014156 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.014116 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" event={"ID":"9d9436d4-9319-4945-bcb7-b476d9f8e775","Type":"ContainerStarted","Data":"96ea499d0bdab68ece6b02abe3a7e64aadbdb01935f960d448334ce10a69c3c9"} Jun 02 04:49:53.041818 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.041695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nwjq\" (UniqueName: \"kubernetes.io/projected/ed3f990a-7d59-4e86-bc84-25d735db2b81-kube-api-access-9nwjq\") pod \"authorino-7498df8756-9kx9c\" (UID: \"ed3f990a-7d59-4e86-bc84-25d735db2b81\") " pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:49:53.085274 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.085239 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:49:53.144059 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.143531 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9nwjq\" (UniqueName: \"kubernetes.io/projected/ed3f990a-7d59-4e86-bc84-25d735db2b81-kube-api-access-9nwjq\") pod \"authorino-7498df8756-9kx9c\" (UID: \"ed3f990a-7d59-4e86-bc84-25d735db2b81\") " pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:49:53.163704 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.163648 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nwjq\" (UniqueName: \"kubernetes.io/projected/ed3f990a-7d59-4e86-bc84-25d735db2b81-kube-api-access-9nwjq\") pod \"authorino-7498df8756-9kx9c\" (UID: \"ed3f990a-7d59-4e86-bc84-25d735db2b81\") " pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:49:53.253863 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.253398 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:49:53.272050 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.272002 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mg9xz"] Jun 02 04:49:53.455127 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:53.455066 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-9kx9c"] Jun 02 04:49:53.458359 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:49:53.457074 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded3f990a_7d59_4e86_bc84_25d735db2b81.slice/crio-4b0e6fa1396e713d7848eae9d5e95f44fff4be98f00c2ab9d9a91350cf2f6b9a WatchSource:0}: Error finding container 4b0e6fa1396e713d7848eae9d5e95f44fff4be98f00c2ab9d9a91350cf2f6b9a: Status 404 returned error can't find the container with id 4b0e6fa1396e713d7848eae9d5e95f44fff4be98f00c2ab9d9a91350cf2f6b9a Jun 02 04:49:54.027469 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:54.027404 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9kx9c" event={"ID":"ed3f990a-7d59-4e86-bc84-25d735db2b81","Type":"ContainerStarted","Data":"4b0e6fa1396e713d7848eae9d5e95f44fff4be98f00c2ab9d9a91350cf2f6b9a"} Jun 02 04:49:54.029592 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:54.029554 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" event={"ID":"024a291f-8937-402c-86b5-3b3bc991a77e","Type":"ContainerStarted","Data":"3dbe18bbbdc1af53b28e69c0da21e3c09af68e16f091225a4709fc37c8595cf3"} Jun 02 04:49:58.047653 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.047615 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9kx9c" event={"ID":"ed3f990a-7d59-4e86-bc84-25d735db2b81","Type":"ContainerStarted","Data":"e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b"} Jun 02 04:49:58.048929 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.048906 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" event={"ID":"024a291f-8937-402c-86b5-3b3bc991a77e","Type":"ContainerStarted","Data":"304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1"} Jun 02 04:49:58.050281 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.050256 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" event={"ID":"9d9436d4-9319-4945-bcb7-b476d9f8e775","Type":"ContainerStarted","Data":"3adb1054251e240f1fc5a842b4acf02dd858f3e66b4fac1ac546ae0ad69b26d9"} Jun 02 04:49:58.050385 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.050373 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:49:58.064924 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.064878 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-9kx9c" podStartSLOduration=2.206038891 podStartE2EDuration="6.064864092s" podCreationTimestamp="2026-06-02 04:49:52 +0000 UTC" firstStartedPulling="2026-06-02 04:49:53.459005654 +0000 UTC m=+475.882775895" lastFinishedPulling="2026-06-02 04:49:57.317830854 +0000 UTC m=+479.741601096" observedRunningTime="2026-06-02 04:49:58.06288052 +0000 UTC m=+480.486650780" watchObservedRunningTime="2026-06-02 04:49:58.064864092 +0000 UTC m=+480.488634355" Jun 02 04:49:58.079631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.079583 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" podStartSLOduration=2.27718608 podStartE2EDuration="7.079564891s" podCreationTimestamp="2026-06-02 04:49:51 +0000 UTC" firstStartedPulling="2026-06-02 04:49:52.431333787 +0000 UTC m=+474.855104041" lastFinishedPulling="2026-06-02 04:49:57.233712606 +0000 UTC m=+479.657482852" observedRunningTime="2026-06-02 04:49:58.079097623 +0000 UTC m=+480.502867888" watchObservedRunningTime="2026-06-02 04:49:58.079564891 +0000 UTC m=+480.503335156" Jun 02 04:49:58.095626 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.095594 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mg9xz"] Jun 02 04:49:58.105098 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:49:58.105052 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" podStartSLOduration=2.082630281 podStartE2EDuration="6.10503885s" podCreationTimestamp="2026-06-02 04:49:52 +0000 UTC" firstStartedPulling="2026-06-02 04:49:53.278327969 +0000 UTC m=+475.702098221" lastFinishedPulling="2026-06-02 04:49:57.300736545 +0000 UTC m=+479.724506790" observedRunningTime="2026-06-02 04:49:58.103698915 +0000 UTC m=+480.527469177" watchObservedRunningTime="2026-06-02 04:49:58.10503885 +0000 UTC m=+480.528809112" Jun 02 04:50:00.057886 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:00.057840 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" podUID="024a291f-8937-402c-86b5-3b3bc991a77e" containerName="authorino" containerID="cri-o://304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1" gracePeriod=30 Jun 02 04:50:00.301097 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:00.301066 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:50:00.416038 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:00.415994 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4zjg\" (UniqueName: \"kubernetes.io/projected/024a291f-8937-402c-86b5-3b3bc991a77e-kube-api-access-k4zjg\") pod \"024a291f-8937-402c-86b5-3b3bc991a77e\" (UID: \"024a291f-8937-402c-86b5-3b3bc991a77e\") " Jun 02 04:50:00.418070 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:00.418044 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024a291f-8937-402c-86b5-3b3bc991a77e-kube-api-access-k4zjg" (OuterVolumeSpecName: "kube-api-access-k4zjg") pod "024a291f-8937-402c-86b5-3b3bc991a77e" (UID: "024a291f-8937-402c-86b5-3b3bc991a77e"). InnerVolumeSpecName "kube-api-access-k4zjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:50:00.517542 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:00.517495 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4zjg\" (UniqueName: \"kubernetes.io/projected/024a291f-8937-402c-86b5-3b3bc991a77e-kube-api-access-k4zjg\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:50:01.062993 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.062951 2566 generic.go:358] "Generic (PLEG): container finished" podID="024a291f-8937-402c-86b5-3b3bc991a77e" containerID="304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1" exitCode=0 Jun 02 04:50:01.063377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.063002 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" Jun 02 04:50:01.063377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.063001 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" event={"ID":"024a291f-8937-402c-86b5-3b3bc991a77e","Type":"ContainerDied","Data":"304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1"} Jun 02 04:50:01.063377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.063095 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mg9xz" event={"ID":"024a291f-8937-402c-86b5-3b3bc991a77e","Type":"ContainerDied","Data":"3dbe18bbbdc1af53b28e69c0da21e3c09af68e16f091225a4709fc37c8595cf3"} Jun 02 04:50:01.063377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.063114 2566 scope.go:117] "RemoveContainer" containerID="304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1" Jun 02 04:50:01.072204 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.072183 2566 scope.go:117] "RemoveContainer" containerID="304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1" Jun 02 04:50:01.072495 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:50:01.072474 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1\": container with ID starting with 304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1 not found: ID does not exist" containerID="304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1" Jun 02 04:50:01.072546 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.072506 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1"} err="failed to get container status \"304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1\": rpc error: code = NotFound desc = could not find container \"304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1\": container with ID starting with 304c036746dce3c154d25835eb67d149c27fab03ada7dd7c4ef6a5dd44ebf6f1 not found: ID does not exist" Jun 02 04:50:01.085612 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.085574 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mg9xz"] Jun 02 04:50:01.090177 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:01.090128 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mg9xz"] Jun 02 04:50:02.119852 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:02.119820 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="024a291f-8937-402c-86b5-3b3bc991a77e" path="/var/lib/kubelet/pods/024a291f-8937-402c-86b5-3b3bc991a77e/volumes" Jun 02 04:50:06.560496 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:06.560455 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:50:06.561062 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:06.560698 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" podUID="9d9436d4-9319-4945-bcb7-b476d9f8e775" containerName="limitador" containerID="cri-o://3adb1054251e240f1fc5a842b4acf02dd858f3e66b4fac1ac546ae0ad69b26d9" gracePeriod=30 Jun 02 04:50:06.561395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:06.561330 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:50:07.089814 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.089782 2566 generic.go:358] "Generic (PLEG): container finished" podID="9d9436d4-9319-4945-bcb7-b476d9f8e775" containerID="3adb1054251e240f1fc5a842b4acf02dd858f3e66b4fac1ac546ae0ad69b26d9" exitCode=0 Jun 02 04:50:07.089971 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.089832 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" event={"ID":"9d9436d4-9319-4945-bcb7-b476d9f8e775","Type":"ContainerDied","Data":"3adb1054251e240f1fc5a842b4acf02dd858f3e66b4fac1ac546ae0ad69b26d9"} Jun 02 04:50:07.110217 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.110191 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:50:07.175790 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.175692 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7gbq\" (UniqueName: \"kubernetes.io/projected/9d9436d4-9319-4945-bcb7-b476d9f8e775-kube-api-access-k7gbq\") pod \"9d9436d4-9319-4945-bcb7-b476d9f8e775\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " Jun 02 04:50:07.175790 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.175741 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d9436d4-9319-4945-bcb7-b476d9f8e775-config-file\") pod \"9d9436d4-9319-4945-bcb7-b476d9f8e775\" (UID: \"9d9436d4-9319-4945-bcb7-b476d9f8e775\") " Jun 02 04:50:07.176165 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.176117 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d9436d4-9319-4945-bcb7-b476d9f8e775-config-file" (OuterVolumeSpecName: "config-file") pod "9d9436d4-9319-4945-bcb7-b476d9f8e775" (UID: "9d9436d4-9319-4945-bcb7-b476d9f8e775"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:50:07.177818 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.177792 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d9436d4-9319-4945-bcb7-b476d9f8e775-kube-api-access-k7gbq" (OuterVolumeSpecName: "kube-api-access-k7gbq") pod "9d9436d4-9319-4945-bcb7-b476d9f8e775" (UID: "9d9436d4-9319-4945-bcb7-b476d9f8e775"). InnerVolumeSpecName "kube-api-access-k7gbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:50:07.276471 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.276426 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7gbq\" (UniqueName: \"kubernetes.io/projected/9d9436d4-9319-4945-bcb7-b476d9f8e775-kube-api-access-k7gbq\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:50:07.276471 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:07.276465 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d9436d4-9319-4945-bcb7-b476d9f8e775-config-file\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:50:08.094541 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:08.094511 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" Jun 02 04:50:08.094541 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:08.094521 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-xb2qr" event={"ID":"9d9436d4-9319-4945-bcb7-b476d9f8e775","Type":"ContainerDied","Data":"96ea499d0bdab68ece6b02abe3a7e64aadbdb01935f960d448334ce10a69c3c9"} Jun 02 04:50:08.095046 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:08.094564 2566 scope.go:117] "RemoveContainer" containerID="3adb1054251e240f1fc5a842b4acf02dd858f3e66b4fac1ac546ae0ad69b26d9" Jun 02 04:50:08.119572 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:08.119541 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:50:08.122680 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:08.122653 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-xb2qr"] Jun 02 04:50:10.119878 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:10.119845 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d9436d4-9319-4945-bcb7-b476d9f8e775" path="/var/lib/kubelet/pods/9d9436d4-9319-4945-bcb7-b476d9f8e775/volumes" Jun 02 04:50:12.232409 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232377 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-nhjp9"] Jun 02 04:50:12.232864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232712 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9436d4-9319-4945-bcb7-b476d9f8e775" containerName="limitador" Jun 02 04:50:12.232864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232724 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9436d4-9319-4945-bcb7-b476d9f8e775" containerName="limitador" Jun 02 04:50:12.232864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232733 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024a291f-8937-402c-86b5-3b3bc991a77e" containerName="authorino" Jun 02 04:50:12.232864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232739 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="024a291f-8937-402c-86b5-3b3bc991a77e" containerName="authorino" Jun 02 04:50:12.232864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232790 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="024a291f-8937-402c-86b5-3b3bc991a77e" containerName="authorino" Jun 02 04:50:12.232864 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.232802 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="9d9436d4-9319-4945-bcb7-b476d9f8e775" containerName="limitador" Jun 02 04:50:12.236820 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.236800 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.239962 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.239921 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 02 04:50:12.240079 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.240013 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ttfd8\"" Jun 02 04:50:12.251561 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.251533 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-nhjp9"] Jun 02 04:50:12.325115 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.325074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f00d7938-156b-405e-ac52-beae953ccc0e-data\") pod \"postgres-68c4fbbd6f-nhjp9\" (UID: \"f00d7938-156b-405e-ac52-beae953ccc0e\") " pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.325115 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.325118 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cclxm\" (UniqueName: \"kubernetes.io/projected/f00d7938-156b-405e-ac52-beae953ccc0e-kube-api-access-cclxm\") pod \"postgres-68c4fbbd6f-nhjp9\" (UID: \"f00d7938-156b-405e-ac52-beae953ccc0e\") " pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.426272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.426230 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f00d7938-156b-405e-ac52-beae953ccc0e-data\") pod \"postgres-68c4fbbd6f-nhjp9\" (UID: \"f00d7938-156b-405e-ac52-beae953ccc0e\") " pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.426272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.426276 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cclxm\" (UniqueName: \"kubernetes.io/projected/f00d7938-156b-405e-ac52-beae953ccc0e-kube-api-access-cclxm\") pod \"postgres-68c4fbbd6f-nhjp9\" (UID: \"f00d7938-156b-405e-ac52-beae953ccc0e\") " pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.426729 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.426700 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f00d7938-156b-405e-ac52-beae953ccc0e-data\") pod \"postgres-68c4fbbd6f-nhjp9\" (UID: \"f00d7938-156b-405e-ac52-beae953ccc0e\") " pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.435199 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.435172 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cclxm\" (UniqueName: \"kubernetes.io/projected/f00d7938-156b-405e-ac52-beae953ccc0e-kube-api-access-cclxm\") pod \"postgres-68c4fbbd6f-nhjp9\" (UID: \"f00d7938-156b-405e-ac52-beae953ccc0e\") " pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.547252 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.547130 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:12.679350 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:12.679321 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-nhjp9"] Jun 02 04:50:12.680961 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:50:12.680935 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00d7938_156b_405e_ac52_beae953ccc0e.slice/crio-574417b06806f3ee7847f1a988b0be3f3ec5deba8515e8be04054563dbf1737a WatchSource:0}: Error finding container 574417b06806f3ee7847f1a988b0be3f3ec5deba8515e8be04054563dbf1737a: Status 404 returned error can't find the container with id 574417b06806f3ee7847f1a988b0be3f3ec5deba8515e8be04054563dbf1737a Jun 02 04:50:13.114356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:13.114316 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" event={"ID":"f00d7938-156b-405e-ac52-beae953ccc0e","Type":"ContainerStarted","Data":"574417b06806f3ee7847f1a988b0be3f3ec5deba8515e8be04054563dbf1737a"} Jun 02 04:50:15.123157 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:15.123101 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" event={"ID":"f00d7938-156b-405e-ac52-beae953ccc0e","Type":"ContainerStarted","Data":"197cc4b5dbda56ceab7e7a476149670bea09e49a1f1e72939f7c34e0b420d6d8"} Jun 02 04:50:15.123534 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:15.123196 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:15.139888 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:15.139811 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" podStartSLOduration=1.015492491 podStartE2EDuration="3.13979348s" podCreationTimestamp="2026-06-02 04:50:12 +0000 UTC" firstStartedPulling="2026-06-02 04:50:12.68207482 +0000 UTC m=+495.105845061" lastFinishedPulling="2026-06-02 04:50:14.806375803 +0000 UTC m=+497.230146050" observedRunningTime="2026-06-02 04:50:15.139130084 +0000 UTC m=+497.562900348" watchObservedRunningTime="2026-06-02 04:50:15.13979348 +0000 UTC m=+497.563563743" Jun 02 04:50:21.157631 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:21.157602 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-nhjp9" Jun 02 04:50:22.911571 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:22.911535 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm"] Jun 02 04:50:22.917853 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:22.917831 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:22.920742 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:22.920716 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 04:50:22.920916 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:22.920896 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v7hkf\"" Jun 02 04:50:22.921532 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:22.921511 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 04:50:22.924764 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:22.924739 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm"] Jun 02 04:50:23.016833 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.016782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.017019 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.016843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-867kf\" (UniqueName: \"kubernetes.io/projected/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-kube-api-access-867kf\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.017019 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.016885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.117746 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.117704 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.117746 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.117753 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-867kf\" (UniqueName: \"kubernetes.io/projected/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-kube-api-access-867kf\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.117976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.117871 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.118118 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.118095 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-bundle\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.118285 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.118266 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-util\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.126347 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.126324 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-867kf\" (UniqueName: \"kubernetes.io/projected/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-kube-api-access-867kf\") pod \"32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.228597 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.228505 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:23.362025 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:23.361994 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm"] Jun 02 04:50:23.363479 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:50:23.363448 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb4f351_d9bc_45af_a8b6_a93b2a8a764c.slice/crio-9b22d0e525454b90b67b511cac45697756e4cf172bddef5c654c01740be76bf4 WatchSource:0}: Error finding container 9b22d0e525454b90b67b511cac45697756e4cf172bddef5c654c01740be76bf4: Status 404 returned error can't find the container with id 9b22d0e525454b90b67b511cac45697756e4cf172bddef5c654c01740be76bf4 Jun 02 04:50:24.155459 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:24.155423 2566 generic.go:358] "Generic (PLEG): container finished" podID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerID="aedd8453d8331f45d4b4f9c50025f5bc0c77dff21f8c3ba64db2b06a34f18936" exitCode=0 Jun 02 04:50:24.155843 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:24.155505 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" event={"ID":"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c","Type":"ContainerDied","Data":"aedd8453d8331f45d4b4f9c50025f5bc0c77dff21f8c3ba64db2b06a34f18936"} Jun 02 04:50:24.155843 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:24.155539 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" event={"ID":"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c","Type":"ContainerStarted","Data":"9b22d0e525454b90b67b511cac45697756e4cf172bddef5c654c01740be76bf4"} Jun 02 04:50:26.166815 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:26.166776 2566 generic.go:358] "Generic (PLEG): container finished" podID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerID="fd09f3d9ff0363c2fe76356b47160334e4bb67c2f09c0bf2314698e62065c0d4" exitCode=0 Jun 02 04:50:26.167228 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:26.166856 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" event={"ID":"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c","Type":"ContainerDied","Data":"fd09f3d9ff0363c2fe76356b47160334e4bb67c2f09c0bf2314698e62065c0d4"} Jun 02 04:50:27.172693 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:27.172656 2566 generic.go:358] "Generic (PLEG): container finished" podID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerID="6e191049b84bd8007fcd97e4a0d6ef8791d1f84b2d95a2c105a78c2fbce10c71" exitCode=0 Jun 02 04:50:27.173158 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:27.172747 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" event={"ID":"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c","Type":"ContainerDied","Data":"6e191049b84bd8007fcd97e4a0d6ef8791d1f84b2d95a2c105a78c2fbce10c71"} Jun 02 04:50:28.309845 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.309814 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:28.466957 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.466851 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-util\") pod \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " Jun 02 04:50:28.466957 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.466895 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-bundle\") pod \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " Jun 02 04:50:28.466957 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.466918 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-867kf\" (UniqueName: \"kubernetes.io/projected/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-kube-api-access-867kf\") pod \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\" (UID: \"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c\") " Jun 02 04:50:28.467426 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.467396 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-bundle" (OuterVolumeSpecName: "bundle") pod "1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" (UID: "1cb4f351-d9bc-45af-a8b6-a93b2a8a764c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:50:28.469052 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.469028 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-kube-api-access-867kf" (OuterVolumeSpecName: "kube-api-access-867kf") pod "1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" (UID: "1cb4f351-d9bc-45af-a8b6-a93b2a8a764c"). InnerVolumeSpecName "kube-api-access-867kf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:50:28.472415 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.472389 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-util" (OuterVolumeSpecName: "util") pod "1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" (UID: "1cb4f351-d9bc-45af-a8b6-a93b2a8a764c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 04:50:28.567901 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.567845 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-util\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:50:28.567901 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.567897 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:50:28.567901 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:28.567911 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-867kf\" (UniqueName: \"kubernetes.io/projected/1cb4f351-d9bc-45af-a8b6-a93b2a8a764c-kube-api-access-867kf\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:50:29.181434 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:29.181393 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" event={"ID":"1cb4f351-d9bc-45af-a8b6-a93b2a8a764c","Type":"ContainerDied","Data":"9b22d0e525454b90b67b511cac45697756e4cf172bddef5c654c01740be76bf4"} Jun 02 04:50:29.181434 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:29.181428 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b22d0e525454b90b67b511cac45697756e4cf172bddef5c654c01740be76bf4" Jun 02 04:50:29.181434 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:29.181430 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/32684d7096c88e320aaae000f43b8c5ea50d1c3368188ae50aa1502a70w8mwm" Jun 02 04:50:52.124733 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.124652 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125043 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="pull" Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125059 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="pull" Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125069 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="util" Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125075 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="util" Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125083 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="extract" Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125089 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="extract" Jun 02 04:50:52.125311 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.125177 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="1cb4f351-d9bc-45af-a8b6-a93b2a8a764c" containerName="extract" Jun 02 04:50:52.128351 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.128334 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 04:50:52.130717 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.130693 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 04:50:52.130841 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.130714 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-8qfn9\"" Jun 02 04:50:52.131453 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.131429 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 04:50:52.131453 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.131447 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 04:50:52.138009 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.137984 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:50:52.173625 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.173589 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqsgq\" (UniqueName: \"kubernetes.io/projected/13438267-90e5-45b3-85dc-0f70985d4b62-kube-api-access-dqsgq\") pod \"maas-keycloak-0\" (UID: \"13438267-90e5-45b3-85dc-0f70985d4b62\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:50:52.274672 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.274638 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dqsgq\" (UniqueName: \"kubernetes.io/projected/13438267-90e5-45b3-85dc-0f70985d4b62-kube-api-access-dqsgq\") pod \"maas-keycloak-0\" (UID: \"13438267-90e5-45b3-85dc-0f70985d4b62\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:50:52.284433 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.284404 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqsgq\" (UniqueName: \"kubernetes.io/projected/13438267-90e5-45b3-85dc-0f70985d4b62-kube-api-access-dqsgq\") pod \"maas-keycloak-0\" (UID: \"13438267-90e5-45b3-85dc-0f70985d4b62\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:50:52.439374 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.439258 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 04:50:52.574854 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:52.574830 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:50:52.577042 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:50:52.577013 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13438267_90e5_45b3_85dc_0f70985d4b62.slice/crio-85a71f082f719d53e1660d9b30c9432603439ffc19fbf56eee61e1ea2d35b482 WatchSource:0}: Error finding container 85a71f082f719d53e1660d9b30c9432603439ffc19fbf56eee61e1ea2d35b482: Status 404 returned error can't find the container with id 85a71f082f719d53e1660d9b30c9432603439ffc19fbf56eee61e1ea2d35b482 Jun 02 04:50:53.270652 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:53.270553 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"13438267-90e5-45b3-85dc-0f70985d4b62","Type":"ContainerStarted","Data":"85a71f082f719d53e1660d9b30c9432603439ffc19fbf56eee61e1ea2d35b482"} Jun 02 04:50:58.293252 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:58.293202 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"13438267-90e5-45b3-85dc-0f70985d4b62","Type":"ContainerStarted","Data":"80a6ca259b4e2bc93206c351b604769326293f33412389e218d8c61336d36c22"} Jun 02 04:50:58.316932 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:58.316857 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.6208671209999999 podStartE2EDuration="6.316832668s" podCreationTimestamp="2026-06-02 04:50:52 +0000 UTC" firstStartedPulling="2026-06-02 04:50:52.578335572 +0000 UTC m=+535.002105815" lastFinishedPulling="2026-06-02 04:50:57.274301108 +0000 UTC m=+539.698071362" observedRunningTime="2026-06-02 04:50:58.312312393 +0000 UTC m=+540.736082669" watchObservedRunningTime="2026-06-02 04:50:58.316832668 +0000 UTC m=+540.740602932" Jun 02 04:50:58.440310 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:58.440202 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 04:50:58.442064 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:58.442024 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:50:59.440606 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:50:59.440550 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:00.440031 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:00.439973 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:01.440082 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:01.440028 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:02.440405 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:02.440345 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 04:51:02.440933 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:02.440710 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:03.440432 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:03.440373 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:04.439816 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:04.439765 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:05.439915 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:05.439856 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:06.440780 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:06.440724 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:07.439802 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:07.439745 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:08.440688 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:08.440635 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:09.440445 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:09.440391 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.44:9000/health/started\": dial tcp 10.134.0.44:9000: connect: connection refused" Jun 02 04:51:10.536810 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:10.536767 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 04:51:10.553677 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:10.553604 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 04:51:20.543049 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:20.543002 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 04:51:21.493119 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.493078 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-sl89k"] Jun 02 04:51:21.500985 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.500959 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:21.501953 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.501927 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-sl89k"] Jun 02 04:51:21.566119 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.566080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7ghp\" (UniqueName: \"kubernetes.io/projected/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e-kube-api-access-h7ghp\") pod \"authorino-8b475cf9f-sl89k\" (UID: \"7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e\") " pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:21.667103 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.667068 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-sl89k"] Jun 02 04:51:21.667103 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.667093 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h7ghp\" (UniqueName: \"kubernetes.io/projected/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e-kube-api-access-h7ghp\") pod \"authorino-8b475cf9f-sl89k\" (UID: \"7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e\") " pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:21.667375 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:51:21.667275 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-h7ghp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8b475cf9f-sl89k" podUID="7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e" Jun 02 04:51:21.677364 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.677333 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7ghp\" (UniqueName: \"kubernetes.io/projected/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e-kube-api-access-h7ghp\") pod \"authorino-8b475cf9f-sl89k\" (UID: \"7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e\") " pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:21.695850 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.695820 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56fdd757f5-k5hlq"] Jun 02 04:51:21.697902 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.697884 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.700278 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.700261 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 02 04:51:21.710029 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.710000 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-k5hlq"] Jun 02 04:51:21.765299 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.765204 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-k5hlq"] Jun 02 04:51:21.765492 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:51:21.765469 2566 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-p4mbw tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-p4mbw tls-cert]: context canceled" pod="kuadrant-system/authorino-56fdd757f5-k5hlq" podUID="118fe38e-44e5-42d5-85a4-a8f00b066247" Jun 02 04:51:21.768092 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.768067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4mbw\" (UniqueName: \"kubernetes.io/projected/118fe38e-44e5-42d5-85a4-a8f00b066247-kube-api-access-p4mbw\") pod \"authorino-56fdd757f5-k5hlq\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.768216 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.768133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/118fe38e-44e5-42d5-85a4-a8f00b066247-tls-cert\") pod \"authorino-56fdd757f5-k5hlq\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.795314 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.795280 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56b496b8f9-j5qz6"] Jun 02 04:51:21.797646 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.797628 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:21.810407 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.810380 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56b496b8f9-j5qz6"] Jun 02 04:51:21.869570 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.869526 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/118fe38e-44e5-42d5-85a4-a8f00b066247-tls-cert\") pod \"authorino-56fdd757f5-k5hlq\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.869746 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.869583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fe363dec-4af5-4cf7-8a57-84e214dcbf08-tls-cert\") pod \"authorino-56b496b8f9-j5qz6\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:21.869746 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.869648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2ptn\" (UniqueName: \"kubernetes.io/projected/fe363dec-4af5-4cf7-8a57-84e214dcbf08-kube-api-access-v2ptn\") pod \"authorino-56b496b8f9-j5qz6\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:21.869746 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.869675 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p4mbw\" (UniqueName: \"kubernetes.io/projected/118fe38e-44e5-42d5-85a4-a8f00b066247-kube-api-access-p4mbw\") pod \"authorino-56fdd757f5-k5hlq\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.872181 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.872129 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/118fe38e-44e5-42d5-85a4-a8f00b066247-tls-cert\") pod \"authorino-56fdd757f5-k5hlq\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.878448 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.878421 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4mbw\" (UniqueName: \"kubernetes.io/projected/118fe38e-44e5-42d5-85a4-a8f00b066247-kube-api-access-p4mbw\") pod \"authorino-56fdd757f5-k5hlq\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:21.970442 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.970400 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fe363dec-4af5-4cf7-8a57-84e214dcbf08-tls-cert\") pod \"authorino-56b496b8f9-j5qz6\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:21.970617 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.970463 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v2ptn\" (UniqueName: \"kubernetes.io/projected/fe363dec-4af5-4cf7-8a57-84e214dcbf08-kube-api-access-v2ptn\") pod \"authorino-56b496b8f9-j5qz6\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:21.972926 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.972899 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fe363dec-4af5-4cf7-8a57-84e214dcbf08-tls-cert\") pod \"authorino-56b496b8f9-j5qz6\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:21.978264 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:21.978236 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2ptn\" (UniqueName: \"kubernetes.io/projected/fe363dec-4af5-4cf7-8a57-84e214dcbf08-kube-api-access-v2ptn\") pod \"authorino-56b496b8f9-j5qz6\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:22.106999 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.106960 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:51:22.255612 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.255581 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56b496b8f9-j5qz6"] Jun 02 04:51:22.257019 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:51:22.256994 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe363dec_4af5_4cf7_8a57_84e214dcbf08.slice/crio-a9c86425761d3ee69d5dd55a51325f075144d4b4bb428454b3fcb7bfdab8af83 WatchSource:0}: Error finding container a9c86425761d3ee69d5dd55a51325f075144d4b4bb428454b3fcb7bfdab8af83: Status 404 returned error can't find the container with id a9c86425761d3ee69d5dd55a51325f075144d4b4bb428454b3fcb7bfdab8af83 Jun 02 04:51:22.407218 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.407107 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" event={"ID":"fe363dec-4af5-4cf7-8a57-84e214dcbf08","Type":"ContainerStarted","Data":"a9c86425761d3ee69d5dd55a51325f075144d4b4bb428454b3fcb7bfdab8af83"} Jun 02 04:51:22.407218 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.407205 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:22.407413 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.407207 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:22.412601 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.412578 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:22.415829 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.415811 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:22.477493 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.477461 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4mbw\" (UniqueName: \"kubernetes.io/projected/118fe38e-44e5-42d5-85a4-a8f00b066247-kube-api-access-p4mbw\") pod \"118fe38e-44e5-42d5-85a4-a8f00b066247\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " Jun 02 04:51:22.477683 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.477575 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/118fe38e-44e5-42d5-85a4-a8f00b066247-tls-cert\") pod \"118fe38e-44e5-42d5-85a4-a8f00b066247\" (UID: \"118fe38e-44e5-42d5-85a4-a8f00b066247\") " Jun 02 04:51:22.477683 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.477621 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7ghp\" (UniqueName: \"kubernetes.io/projected/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e-kube-api-access-h7ghp\") pod \"7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e\" (UID: \"7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e\") " Jun 02 04:51:22.479737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.479705 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118fe38e-44e5-42d5-85a4-a8f00b066247-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "118fe38e-44e5-42d5-85a4-a8f00b066247" (UID: "118fe38e-44e5-42d5-85a4-a8f00b066247"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:51:22.479737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.479718 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e-kube-api-access-h7ghp" (OuterVolumeSpecName: "kube-api-access-h7ghp") pod "7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e" (UID: "7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e"). InnerVolumeSpecName "kube-api-access-h7ghp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:51:22.479737 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.479726 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/118fe38e-44e5-42d5-85a4-a8f00b066247-kube-api-access-p4mbw" (OuterVolumeSpecName: "kube-api-access-p4mbw") pod "118fe38e-44e5-42d5-85a4-a8f00b066247" (UID: "118fe38e-44e5-42d5-85a4-a8f00b066247"). InnerVolumeSpecName "kube-api-access-p4mbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:51:22.578377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.578336 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/118fe38e-44e5-42d5-85a4-a8f00b066247-tls-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:51:22.578377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.578376 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7ghp\" (UniqueName: \"kubernetes.io/projected/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e-kube-api-access-h7ghp\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:51:22.578377 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:22.578389 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4mbw\" (UniqueName: \"kubernetes.io/projected/118fe38e-44e5-42d5-85a4-a8f00b066247-kube-api-access-p4mbw\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:51:23.412195 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.412121 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" event={"ID":"fe363dec-4af5-4cf7-8a57-84e214dcbf08","Type":"ContainerStarted","Data":"a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56"} Jun 02 04:51:23.412400 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.412265 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-k5hlq" Jun 02 04:51:23.412400 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.412371 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-sl89k" Jun 02 04:51:23.428073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.428026 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" podStartSLOduration=2.077347476 podStartE2EDuration="2.428012334s" podCreationTimestamp="2026-06-02 04:51:21 +0000 UTC" firstStartedPulling="2026-06-02 04:51:22.258207301 +0000 UTC m=+564.681977542" lastFinishedPulling="2026-06-02 04:51:22.608872155 +0000 UTC m=+565.032642400" observedRunningTime="2026-06-02 04:51:23.426976363 +0000 UTC m=+565.850746626" watchObservedRunningTime="2026-06-02 04:51:23.428012334 +0000 UTC m=+565.851782596" Jun 02 04:51:23.453275 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.453240 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-sl89k"] Jun 02 04:51:23.456808 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.456754 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-9kx9c"] Jun 02 04:51:23.457078 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.457046 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-9kx9c" podUID="ed3f990a-7d59-4e86-bc84-25d735db2b81" containerName="authorino" containerID="cri-o://e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b" gracePeriod=30 Jun 02 04:51:23.460939 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.460716 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-sl89k"] Jun 02 04:51:23.487061 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.487028 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-k5hlq"] Jun 02 04:51:23.489459 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.489429 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-k5hlq"] Jun 02 04:51:23.706375 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.706348 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:51:23.787239 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.787203 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nwjq\" (UniqueName: \"kubernetes.io/projected/ed3f990a-7d59-4e86-bc84-25d735db2b81-kube-api-access-9nwjq\") pod \"ed3f990a-7d59-4e86-bc84-25d735db2b81\" (UID: \"ed3f990a-7d59-4e86-bc84-25d735db2b81\") " Jun 02 04:51:23.789257 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.789225 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3f990a-7d59-4e86-bc84-25d735db2b81-kube-api-access-9nwjq" (OuterVolumeSpecName: "kube-api-access-9nwjq") pod "ed3f990a-7d59-4e86-bc84-25d735db2b81" (UID: "ed3f990a-7d59-4e86-bc84-25d735db2b81"). InnerVolumeSpecName "kube-api-access-9nwjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:51:23.887778 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:23.887744 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nwjq\" (UniqueName: \"kubernetes.io/projected/ed3f990a-7d59-4e86-bc84-25d735db2b81-kube-api-access-9nwjq\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:51:24.119883 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.119847 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="118fe38e-44e5-42d5-85a4-a8f00b066247" path="/var/lib/kubelet/pods/118fe38e-44e5-42d5-85a4-a8f00b066247/volumes" Jun 02 04:51:24.120091 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.120078 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e" path="/var/lib/kubelet/pods/7e9b786a-dd47-472b-ba6c-8ccb5eec4c8e/volumes" Jun 02 04:51:24.417481 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.417384 2566 generic.go:358] "Generic (PLEG): container finished" podID="ed3f990a-7d59-4e86-bc84-25d735db2b81" containerID="e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b" exitCode=0 Jun 02 04:51:24.417481 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.417431 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9kx9c" Jun 02 04:51:24.417481 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.417463 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9kx9c" event={"ID":"ed3f990a-7d59-4e86-bc84-25d735db2b81","Type":"ContainerDied","Data":"e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b"} Jun 02 04:51:24.417731 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.417501 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9kx9c" event={"ID":"ed3f990a-7d59-4e86-bc84-25d735db2b81","Type":"ContainerDied","Data":"4b0e6fa1396e713d7848eae9d5e95f44fff4be98f00c2ab9d9a91350cf2f6b9a"} Jun 02 04:51:24.417731 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.417516 2566 scope.go:117] "RemoveContainer" containerID="e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b" Jun 02 04:51:24.425848 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.425831 2566 scope.go:117] "RemoveContainer" containerID="e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b" Jun 02 04:51:24.426122 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:51:24.426103 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b\": container with ID starting with e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b not found: ID does not exist" containerID="e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b" Jun 02 04:51:24.426412 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.426132 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b"} err="failed to get container status \"e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b\": rpc error: code = NotFound desc = could not find container \"e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b\": container with ID starting with e6560fab75a34a5e02390004fb310de48e39654aebec90811416d731ca4f556b not found: ID does not exist" Jun 02 04:51:24.433919 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.433894 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-9kx9c"] Jun 02 04:51:24.440530 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:24.440505 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-9kx9c"] Jun 02 04:51:26.120021 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.119986 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3f990a-7d59-4e86-bc84-25d735db2b81" path="/var/lib/kubelet/pods/ed3f990a-7d59-4e86-bc84-25d735db2b81/volumes" Jun 02 04:51:26.965347 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.965307 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66d4846f58-bsm7d"] Jun 02 04:51:26.965669 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.965657 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3f990a-7d59-4e86-bc84-25d735db2b81" containerName="authorino" Jun 02 04:51:26.965714 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.965671 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3f990a-7d59-4e86-bc84-25d735db2b81" containerName="authorino" Jun 02 04:51:26.965749 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.965720 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="ed3f990a-7d59-4e86-bc84-25d735db2b81" containerName="authorino" Jun 02 04:51:26.969993 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.969971 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:26.976019 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.975996 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7m59l\"" Jun 02 04:51:26.982702 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.982675 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 04:51:26.985491 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:26.985460 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66d4846f58-bsm7d"] Jun 02 04:51:27.014518 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.014483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sqrb\" (UniqueName: \"kubernetes.io/projected/2908dab0-204a-4ada-9489-c58eff5e3cde-kube-api-access-7sqrb\") pod \"maas-controller-66d4846f58-bsm7d\" (UID: \"2908dab0-204a-4ada-9489-c58eff5e3cde\") " pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:27.115254 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.115214 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7sqrb\" (UniqueName: \"kubernetes.io/projected/2908dab0-204a-4ada-9489-c58eff5e3cde-kube-api-access-7sqrb\") pod \"maas-controller-66d4846f58-bsm7d\" (UID: \"2908dab0-204a-4ada-9489-c58eff5e3cde\") " pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:27.125166 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.125104 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sqrb\" (UniqueName: \"kubernetes.io/projected/2908dab0-204a-4ada-9489-c58eff5e3cde-kube-api-access-7sqrb\") pod \"maas-controller-66d4846f58-bsm7d\" (UID: \"2908dab0-204a-4ada-9489-c58eff5e3cde\") " pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:27.183511 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.183473 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-bt9j9"] Jun 02 04:51:27.186859 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.186841 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:27.195121 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.195090 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-bt9j9"] Jun 02 04:51:27.215984 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.215882 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwks7\" (UniqueName: \"kubernetes.io/projected/faad3b49-e698-4e84-af1b-cf8863661376-kube-api-access-gwks7\") pod \"maas-controller-7cf7f788f6-bt9j9\" (UID: \"faad3b49-e698-4e84-af1b-cf8863661376\") " pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:27.279824 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.279779 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:27.316680 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.316645 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gwks7\" (UniqueName: \"kubernetes.io/projected/faad3b49-e698-4e84-af1b-cf8863661376-kube-api-access-gwks7\") pod \"maas-controller-7cf7f788f6-bt9j9\" (UID: \"faad3b49-e698-4e84-af1b-cf8863661376\") " pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:27.325475 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.325441 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwks7\" (UniqueName: \"kubernetes.io/projected/faad3b49-e698-4e84-af1b-cf8863661376-kube-api-access-gwks7\") pod \"maas-controller-7cf7f788f6-bt9j9\" (UID: \"faad3b49-e698-4e84-af1b-cf8863661376\") " pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:27.413324 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.413295 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66d4846f58-bsm7d"] Jun 02 04:51:27.415340 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:51:27.415310 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2908dab0_204a_4ada_9489_c58eff5e3cde.slice/crio-0722bacd284cd8165eae0d2a2234ad7c8e0ff62c800b903480fd23cd90a9fcbe WatchSource:0}: Error finding container 0722bacd284cd8165eae0d2a2234ad7c8e0ff62c800b903480fd23cd90a9fcbe: Status 404 returned error can't find the container with id 0722bacd284cd8165eae0d2a2234ad7c8e0ff62c800b903480fd23cd90a9fcbe Jun 02 04:51:27.431690 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.431659 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66d4846f58-bsm7d" event={"ID":"2908dab0-204a-4ada-9489-c58eff5e3cde","Type":"ContainerStarted","Data":"0722bacd284cd8165eae0d2a2234ad7c8e0ff62c800b903480fd23cd90a9fcbe"} Jun 02 04:51:27.498603 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.498503 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:27.628803 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:27.628779 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-bt9j9"] Jun 02 04:51:27.630517 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:51:27.630486 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaad3b49_e698_4e84_af1b_cf8863661376.slice/crio-e2697d96cbd8874badd7f5571c999e8888a09927ff913e5a6a908ec5972a6260 WatchSource:0}: Error finding container e2697d96cbd8874badd7f5571c999e8888a09927ff913e5a6a908ec5972a6260: Status 404 returned error can't find the container with id e2697d96cbd8874badd7f5571c999e8888a09927ff913e5a6a908ec5972a6260 Jun 02 04:51:28.441221 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:28.441181 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" event={"ID":"faad3b49-e698-4e84-af1b-cf8863661376","Type":"ContainerStarted","Data":"e2697d96cbd8874badd7f5571c999e8888a09927ff913e5a6a908ec5972a6260"} Jun 02 04:51:30.455611 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:30.455505 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" event={"ID":"faad3b49-e698-4e84-af1b-cf8863661376","Type":"ContainerStarted","Data":"e0d5c347d6d9d23cef955682192d0fa711daa7942647d211e5208879711bf422"} Jun 02 04:51:30.455611 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:30.455566 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:30.456952 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:30.456923 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66d4846f58-bsm7d" event={"ID":"2908dab0-204a-4ada-9489-c58eff5e3cde","Type":"ContainerStarted","Data":"18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c"} Jun 02 04:51:30.457071 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:30.457044 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:30.475427 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:30.475378 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" podStartSLOduration=0.973238507 podStartE2EDuration="3.475366668s" podCreationTimestamp="2026-06-02 04:51:27 +0000 UTC" firstStartedPulling="2026-06-02 04:51:27.631788671 +0000 UTC m=+570.055558915" lastFinishedPulling="2026-06-02 04:51:30.133916821 +0000 UTC m=+572.557687076" observedRunningTime="2026-06-02 04:51:30.474668189 +0000 UTC m=+572.898438451" watchObservedRunningTime="2026-06-02 04:51:30.475366668 +0000 UTC m=+572.899136941" Jun 02 04:51:30.501156 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:30.501069 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-66d4846f58-bsm7d" podStartSLOduration=1.777104746 podStartE2EDuration="4.501053371s" podCreationTimestamp="2026-06-02 04:51:26 +0000 UTC" firstStartedPulling="2026-06-02 04:51:27.416894621 +0000 UTC m=+569.840664865" lastFinishedPulling="2026-06-02 04:51:30.14084325 +0000 UTC m=+572.564613490" observedRunningTime="2026-06-02 04:51:30.499263784 +0000 UTC m=+572.923034048" watchObservedRunningTime="2026-06-02 04:51:30.501053371 +0000 UTC m=+572.924823641" Jun 02 04:51:41.466477 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.466441 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:41.466974 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.466713 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:51:41.521881 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.521841 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66d4846f58-bsm7d"] Jun 02 04:51:41.522101 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.522046 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-66d4846f58-bsm7d" podUID="2908dab0-204a-4ada-9489-c58eff5e3cde" containerName="manager" containerID="cri-o://18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c" gracePeriod=30 Jun 02 04:51:41.761857 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.761829 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:41.848271 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.848240 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sqrb\" (UniqueName: \"kubernetes.io/projected/2908dab0-204a-4ada-9489-c58eff5e3cde-kube-api-access-7sqrb\") pod \"2908dab0-204a-4ada-9489-c58eff5e3cde\" (UID: \"2908dab0-204a-4ada-9489-c58eff5e3cde\") " Jun 02 04:51:41.850546 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.850516 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2908dab0-204a-4ada-9489-c58eff5e3cde-kube-api-access-7sqrb" (OuterVolumeSpecName: "kube-api-access-7sqrb") pod "2908dab0-204a-4ada-9489-c58eff5e3cde" (UID: "2908dab0-204a-4ada-9489-c58eff5e3cde"). InnerVolumeSpecName "kube-api-access-7sqrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:51:41.949132 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:41.949079 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sqrb\" (UniqueName: \"kubernetes.io/projected/2908dab0-204a-4ada-9489-c58eff5e3cde-kube-api-access-7sqrb\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:51:42.504956 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.504854 2566 generic.go:358] "Generic (PLEG): container finished" podID="2908dab0-204a-4ada-9489-c58eff5e3cde" containerID="18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c" exitCode=0 Jun 02 04:51:42.504956 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.504916 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66d4846f58-bsm7d" Jun 02 04:51:42.504956 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.504941 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66d4846f58-bsm7d" event={"ID":"2908dab0-204a-4ada-9489-c58eff5e3cde","Type":"ContainerDied","Data":"18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c"} Jun 02 04:51:42.505493 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.504997 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66d4846f58-bsm7d" event={"ID":"2908dab0-204a-4ada-9489-c58eff5e3cde","Type":"ContainerDied","Data":"0722bacd284cd8165eae0d2a2234ad7c8e0ff62c800b903480fd23cd90a9fcbe"} Jun 02 04:51:42.505493 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.505017 2566 scope.go:117] "RemoveContainer" containerID="18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c" Jun 02 04:51:42.514110 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.514088 2566 scope.go:117] "RemoveContainer" containerID="18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c" Jun 02 04:51:42.514398 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:51:42.514380 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c\": container with ID starting with 18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c not found: ID does not exist" containerID="18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c" Jun 02 04:51:42.514442 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.514408 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c"} err="failed to get container status \"18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c\": rpc error: code = NotFound desc = could not find container \"18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c\": container with ID starting with 18bf11a113be32a0b0c7e7585928ffe67fd7ff8da76c70c7b042c100bc7d508c not found: ID does not exist" Jun 02 04:51:42.522065 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.522035 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66d4846f58-bsm7d"] Jun 02 04:51:42.529067 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:42.529041 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66d4846f58-bsm7d"] Jun 02 04:51:44.119335 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:44.119299 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2908dab0-204a-4ada-9489-c58eff5e3cde" path="/var/lib/kubelet/pods/2908dab0-204a-4ada-9489-c58eff5e3cde/volumes" Jun 02 04:51:58.079409 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:58.079381 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:51:58.080695 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:51:58.080670 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:52:13.811603 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:13.811519 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:52:13.814315 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:13.811768 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" containerID="cri-o://80a6ca259b4e2bc93206c351b604769326293f33412389e218d8c61336d36c22" gracePeriod=30 Jun 02 04:52:15.637921 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:15.637886 2566 generic.go:358] "Generic (PLEG): container finished" podID="13438267-90e5-45b3-85dc-0f70985d4b62" containerID="80a6ca259b4e2bc93206c351b604769326293f33412389e218d8c61336d36c22" exitCode=143 Jun 02 04:52:15.638332 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:15.637957 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"13438267-90e5-45b3-85dc-0f70985d4b62","Type":"ContainerDied","Data":"80a6ca259b4e2bc93206c351b604769326293f33412389e218d8c61336d36c22"} Jun 02 04:52:15.854074 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:15.854042 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:15.941073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:15.940971 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqsgq\" (UniqueName: \"kubernetes.io/projected/13438267-90e5-45b3-85dc-0f70985d4b62-kube-api-access-dqsgq\") pod \"13438267-90e5-45b3-85dc-0f70985d4b62\" (UID: \"13438267-90e5-45b3-85dc-0f70985d4b62\") " Jun 02 04:52:15.943154 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:15.943113 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13438267-90e5-45b3-85dc-0f70985d4b62-kube-api-access-dqsgq" (OuterVolumeSpecName: "kube-api-access-dqsgq") pod "13438267-90e5-45b3-85dc-0f70985d4b62" (UID: "13438267-90e5-45b3-85dc-0f70985d4b62"). InnerVolumeSpecName "kube-api-access-dqsgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:52:16.041860 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.041819 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqsgq\" (UniqueName: \"kubernetes.io/projected/13438267-90e5-45b3-85dc-0f70985d4b62-kube-api-access-dqsgq\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:52:16.643046 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.643013 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.643046 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.643039 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"13438267-90e5-45b3-85dc-0f70985d4b62","Type":"ContainerDied","Data":"85a71f082f719d53e1660d9b30c9432603439ffc19fbf56eee61e1ea2d35b482"} Jun 02 04:52:16.643550 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.643087 2566 scope.go:117] "RemoveContainer" containerID="80a6ca259b4e2bc93206c351b604769326293f33412389e218d8c61336d36c22" Jun 02 04:52:16.680222 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.680179 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:52:16.698005 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.697977 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:52:16.735511 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.735470 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:52:16.736024 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.736005 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" Jun 02 04:52:16.736089 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.736027 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" Jun 02 04:52:16.736089 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.736056 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2908dab0-204a-4ada-9489-c58eff5e3cde" containerName="manager" Jun 02 04:52:16.736089 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.736066 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2908dab0-204a-4ada-9489-c58eff5e3cde" containerName="manager" Jun 02 04:52:16.736224 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.736211 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="2908dab0-204a-4ada-9489-c58eff5e3cde" containerName="manager" Jun 02 04:52:16.736272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.736237 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" containerName="keycloak" Jun 02 04:52:16.740977 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.740955 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.743867 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.743841 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 04:52:16.743997 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.743842 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-8qfn9\"" Jun 02 04:52:16.744080 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.744048 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 04:52:16.744176 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.744159 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 04:52:16.744315 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.744297 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 04:52:16.745976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.745954 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/692cd340-a0c3-4671-9b78-9a87737e741f-test-realms\") pod \"maas-keycloak-0\" (UID: \"692cd340-a0c3-4671-9b78-9a87737e741f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.746270 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.746014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5v2j\" (UniqueName: \"kubernetes.io/projected/692cd340-a0c3-4671-9b78-9a87737e741f-kube-api-access-v5v2j\") pod \"maas-keycloak-0\" (UID: \"692cd340-a0c3-4671-9b78-9a87737e741f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.749870 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.749843 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:52:16.846846 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.846811 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/692cd340-a0c3-4671-9b78-9a87737e741f-test-realms\") pod \"maas-keycloak-0\" (UID: \"692cd340-a0c3-4671-9b78-9a87737e741f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.847025 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.846867 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v5v2j\" (UniqueName: \"kubernetes.io/projected/692cd340-a0c3-4671-9b78-9a87737e741f-kube-api-access-v5v2j\") pod \"maas-keycloak-0\" (UID: \"692cd340-a0c3-4671-9b78-9a87737e741f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.847601 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.847578 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/692cd340-a0c3-4671-9b78-9a87737e741f-test-realms\") pod \"maas-keycloak-0\" (UID: \"692cd340-a0c3-4671-9b78-9a87737e741f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:16.856482 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:16.856455 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5v2j\" (UniqueName: \"kubernetes.io/projected/692cd340-a0c3-4671-9b78-9a87737e741f-kube-api-access-v5v2j\") pod \"maas-keycloak-0\" (UID: \"692cd340-a0c3-4671-9b78-9a87737e741f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:17.059993 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:17.059898 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:17.194581 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:17.194553 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 04:52:17.196191 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:52:17.196160 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692cd340_a0c3_4671_9b78_9a87737e741f.slice/crio-8f601370e4183852b1701fb77b43eee41c5aa0960fdac27077c2c5dc98d6f982 WatchSource:0}: Error finding container 8f601370e4183852b1701fb77b43eee41c5aa0960fdac27077c2c5dc98d6f982: Status 404 returned error can't find the container with id 8f601370e4183852b1701fb77b43eee41c5aa0960fdac27077c2c5dc98d6f982 Jun 02 04:52:17.197446 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:17.197428 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 04:52:17.647870 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:17.647829 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"692cd340-a0c3-4671-9b78-9a87737e741f","Type":"ContainerStarted","Data":"8f601370e4183852b1701fb77b43eee41c5aa0960fdac27077c2c5dc98d6f982"} Jun 02 04:52:18.122665 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:18.122618 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13438267-90e5-45b3-85dc-0f70985d4b62" path="/var/lib/kubelet/pods/13438267-90e5-45b3-85dc-0f70985d4b62/volumes" Jun 02 04:52:18.657331 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:18.657289 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"692cd340-a0c3-4671-9b78-9a87737e741f","Type":"ContainerStarted","Data":"4b011bf4e553d85eb5736d0fb742e5422917339c6e8ef3ceda242e5b461b4d12"} Jun 02 04:52:18.682035 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:18.681949 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.263869863 podStartE2EDuration="2.681914863s" podCreationTimestamp="2026-06-02 04:52:16 +0000 UTC" firstStartedPulling="2026-06-02 04:52:17.197556417 +0000 UTC m=+619.621326658" lastFinishedPulling="2026-06-02 04:52:17.615601406 +0000 UTC m=+620.039371658" observedRunningTime="2026-06-02 04:52:18.678035473 +0000 UTC m=+621.101805736" watchObservedRunningTime="2026-06-02 04:52:18.681914863 +0000 UTC m=+621.105685127" Jun 02 04:52:19.061157 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:19.061033 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:19.063031 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:19.062984 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:20.060759 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:20.060703 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:21.060624 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:21.060568 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:22.061227 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:22.061172 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:23.061172 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:23.061101 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:24.061331 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:24.061274 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:25.060982 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:25.060926 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:26.060690 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:26.060632 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:27.060753 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:27.060707 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:27.061251 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:27.061025 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:28.061367 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:28.061314 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:29.060427 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:29.060377 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:30.061276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:30.061222 2566 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.50:9000/health/started\": dial tcp 10.134.0.50:9000: connect: connection refused" Jun 02 04:52:31.157354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:31.157307 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:31.177395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:31.177059 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="692cd340-a0c3-4671-9b78-9a87737e741f" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 04:52:41.163151 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:41.163100 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 04:52:51.850580 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.850542 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68cfdffbf-mx29n"] Jun 02 04:52:51.854268 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.854246 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:51.856821 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.856800 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 02 04:52:51.869831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.869807 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68cfdffbf-mx29n"] Jun 02 04:52:51.997303 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.997254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f9e8079-b30d-4511-9be1-69bb68cbdb22-tls-cert\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:51.997463 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.997312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpp9f\" (UniqueName: \"kubernetes.io/projected/5f9e8079-b30d-4511-9be1-69bb68cbdb22-kube-api-access-gpp9f\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:51.997463 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:51.997401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5f9e8079-b30d-4511-9be1-69bb68cbdb22-oidc-ca\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.097949 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.097905 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5f9e8079-b30d-4511-9be1-69bb68cbdb22-oidc-ca\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.098134 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.098006 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f9e8079-b30d-4511-9be1-69bb68cbdb22-tls-cert\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.098134 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.098024 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpp9f\" (UniqueName: \"kubernetes.io/projected/5f9e8079-b30d-4511-9be1-69bb68cbdb22-kube-api-access-gpp9f\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.098551 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.098528 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5f9e8079-b30d-4511-9be1-69bb68cbdb22-oidc-ca\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.100443 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.100419 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f9e8079-b30d-4511-9be1-69bb68cbdb22-tls-cert\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.107249 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.107194 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpp9f\" (UniqueName: \"kubernetes.io/projected/5f9e8079-b30d-4511-9be1-69bb68cbdb22-kube-api-access-gpp9f\") pod \"authorino-68cfdffbf-mx29n\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.163012 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.162957 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:52:52.293557 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.293520 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68cfdffbf-mx29n"] Jun 02 04:52:52.295067 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:52:52.295039 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f9e8079_b30d_4511_9be1_69bb68cbdb22.slice/crio-97860e91d145c33a38b2074fae7631f50076ae5851d8c51dd279dc91e8528349 WatchSource:0}: Error finding container 97860e91d145c33a38b2074fae7631f50076ae5851d8c51dd279dc91e8528349: Status 404 returned error can't find the container with id 97860e91d145c33a38b2074fae7631f50076ae5851d8c51dd279dc91e8528349 Jun 02 04:52:52.820886 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.820843 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68cfdffbf-mx29n" event={"ID":"5f9e8079-b30d-4511-9be1-69bb68cbdb22","Type":"ContainerStarted","Data":"f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0"} Jun 02 04:52:52.820886 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.820886 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68cfdffbf-mx29n" event={"ID":"5f9e8079-b30d-4511-9be1-69bb68cbdb22","Type":"ContainerStarted","Data":"97860e91d145c33a38b2074fae7631f50076ae5851d8c51dd279dc91e8528349"} Jun 02 04:52:52.840579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.840523 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-68cfdffbf-mx29n" podStartSLOduration=1.455986724 podStartE2EDuration="1.840504086s" podCreationTimestamp="2026-06-02 04:52:51 +0000 UTC" firstStartedPulling="2026-06-02 04:52:52.296373412 +0000 UTC m=+654.720143656" lastFinishedPulling="2026-06-02 04:52:52.680890777 +0000 UTC m=+655.104661018" observedRunningTime="2026-06-02 04:52:52.839806461 +0000 UTC m=+655.263576724" watchObservedRunningTime="2026-06-02 04:52:52.840504086 +0000 UTC m=+655.264274344" Jun 02 04:52:52.875979 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.875934 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56b496b8f9-j5qz6"] Jun 02 04:52:52.876427 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:52.876169 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" podUID="fe363dec-4af5-4cf7-8a57-84e214dcbf08" containerName="authorino" containerID="cri-o://a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56" gracePeriod=30 Jun 02 04:52:53.140619 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.140591 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:52:53.205464 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.205424 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2ptn\" (UniqueName: \"kubernetes.io/projected/fe363dec-4af5-4cf7-8a57-84e214dcbf08-kube-api-access-v2ptn\") pod \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " Jun 02 04:52:53.205654 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.205531 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fe363dec-4af5-4cf7-8a57-84e214dcbf08-tls-cert\") pod \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\" (UID: \"fe363dec-4af5-4cf7-8a57-84e214dcbf08\") " Jun 02 04:52:53.208243 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.208196 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe363dec-4af5-4cf7-8a57-84e214dcbf08-kube-api-access-v2ptn" (OuterVolumeSpecName: "kube-api-access-v2ptn") pod "fe363dec-4af5-4cf7-8a57-84e214dcbf08" (UID: "fe363dec-4af5-4cf7-8a57-84e214dcbf08"). InnerVolumeSpecName "kube-api-access-v2ptn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:52:53.216908 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.216864 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe363dec-4af5-4cf7-8a57-84e214dcbf08-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "fe363dec-4af5-4cf7-8a57-84e214dcbf08" (UID: "fe363dec-4af5-4cf7-8a57-84e214dcbf08"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:52:53.306407 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.306321 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fe363dec-4af5-4cf7-8a57-84e214dcbf08-tls-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:52:53.306407 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.306354 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2ptn\" (UniqueName: \"kubernetes.io/projected/fe363dec-4af5-4cf7-8a57-84e214dcbf08-kube-api-access-v2ptn\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:52:53.825920 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.825882 2566 generic.go:358] "Generic (PLEG): container finished" podID="fe363dec-4af5-4cf7-8a57-84e214dcbf08" containerID="a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56" exitCode=0 Jun 02 04:52:53.826128 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.825936 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" Jun 02 04:52:53.826128 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.825978 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" event={"ID":"fe363dec-4af5-4cf7-8a57-84e214dcbf08","Type":"ContainerDied","Data":"a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56"} Jun 02 04:52:53.826128 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.826023 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b496b8f9-j5qz6" event={"ID":"fe363dec-4af5-4cf7-8a57-84e214dcbf08","Type":"ContainerDied","Data":"a9c86425761d3ee69d5dd55a51325f075144d4b4bb428454b3fcb7bfdab8af83"} Jun 02 04:52:53.826128 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.826043 2566 scope.go:117] "RemoveContainer" containerID="a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56" Jun 02 04:52:53.834800 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.834783 2566 scope.go:117] "RemoveContainer" containerID="a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56" Jun 02 04:52:53.835082 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:52:53.835065 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56\": container with ID starting with a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56 not found: ID does not exist" containerID="a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56" Jun 02 04:52:53.835133 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.835091 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56"} err="failed to get container status \"a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56\": rpc error: code = NotFound desc = could not find container \"a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56\": container with ID starting with a0a89630ab4f8e3bfebae6a031cd18847115049ea029b5d25aa708d66a2bbc56 not found: ID does not exist" Jun 02 04:52:53.870910 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.870872 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56b496b8f9-j5qz6"] Jun 02 04:52:53.880309 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:53.880276 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56b496b8f9-j5qz6"] Jun 02 04:52:54.119915 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:52:54.119822 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe363dec-4af5-4cf7-8a57-84e214dcbf08" path="/var/lib/kubelet/pods/fe363dec-4af5-4cf7-8a57-84e214dcbf08/volumes" Jun 02 04:53:20.534712 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.534673 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c"] Jun 02 04:53:20.535295 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.535032 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe363dec-4af5-4cf7-8a57-84e214dcbf08" containerName="authorino" Jun 02 04:53:20.535295 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.535044 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe363dec-4af5-4cf7-8a57-84e214dcbf08" containerName="authorino" Jun 02 04:53:20.535295 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.535107 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="fe363dec-4af5-4cf7-8a57-84e214dcbf08" containerName="authorino" Jun 02 04:53:20.538375 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.538353 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.543748 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.543718 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 04:53:20.544487 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.544463 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 04:53:20.544487 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.544465 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 02 04:53:20.544675 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.544537 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dc2sf\"" Jun 02 04:53:20.550698 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.550675 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c"] Jun 02 04:53:20.643114 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.643075 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.643356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.643174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.643356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.643204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.643356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.643234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.643356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.643256 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f20b02e-46b5-4ebe-b501-a39eea266150-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.643356 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.643330 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mvlq\" (UniqueName: \"kubernetes.io/projected/7f20b02e-46b5-4ebe-b501-a39eea266150-kube-api-access-9mvlq\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.743922 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.743862 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mvlq\" (UniqueName: \"kubernetes.io/projected/7f20b02e-46b5-4ebe-b501-a39eea266150-kube-api-access-9mvlq\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744119 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744028 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744198 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744128 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744198 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744178 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744303 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744223 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744303 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744252 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f20b02e-46b5-4ebe-b501-a39eea266150-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744539 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744506 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744681 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744581 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.744681 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.744651 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.746361 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.746332 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f20b02e-46b5-4ebe-b501-a39eea266150-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.746607 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.746590 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f20b02e-46b5-4ebe-b501-a39eea266150-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.751880 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.751864 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mvlq\" (UniqueName: \"kubernetes.io/projected/7f20b02e-46b5-4ebe-b501-a39eea266150-kube-api-access-9mvlq\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c\" (UID: \"7f20b02e-46b5-4ebe-b501-a39eea266150\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:20.848416 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:20.848362 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:21.004609 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:21.004577 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c"] Jun 02 04:53:21.006282 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:53:21.006245 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f20b02e_46b5_4ebe_b501_a39eea266150.slice/crio-529adafa9bf540e570df7777367b3f4cf76084091ef6be2b1aafd9e85bb5851c WatchSource:0}: Error finding container 529adafa9bf540e570df7777367b3f4cf76084091ef6be2b1aafd9e85bb5851c: Status 404 returned error can't find the container with id 529adafa9bf540e570df7777367b3f4cf76084091ef6be2b1aafd9e85bb5851c Jun 02 04:53:21.938625 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:21.938588 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" event={"ID":"7f20b02e-46b5-4ebe-b501-a39eea266150","Type":"ContainerStarted","Data":"529adafa9bf540e570df7777367b3f4cf76084091ef6be2b1aafd9e85bb5851c"} Jun 02 04:53:25.959370 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:25.959328 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" event={"ID":"7f20b02e-46b5-4ebe-b501-a39eea266150","Type":"ContainerStarted","Data":"e8754b5c05fe9997b2397e6f27c8ff94fbe7aed81424d7cd0042b3ddd7f57ed3"} Jun 02 04:53:26.265939 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.265842 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6"] Jun 02 04:53:26.270760 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.270732 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.274427 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.274400 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 02 04:53:26.287913 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.287832 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6"] Jun 02 04:53:26.398178 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.398126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.398342 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.398198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.398342 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.398275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.398342 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.398318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xx4s\" (UniqueName: \"kubernetes.io/projected/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-kube-api-access-5xx4s\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.398447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.398361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.398447 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.398378 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499321 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499284 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499534 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499332 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5xx4s\" (UniqueName: \"kubernetes.io/projected/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-kube-api-access-5xx4s\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499534 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499369 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499534 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499385 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499534 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499426 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499772 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499747 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.499828 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499762 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.500015 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.499988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.500150 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.500112 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.502169 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.502128 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.502452 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.502435 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.510350 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.510327 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xx4s\" (UniqueName: \"kubernetes.io/projected/6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e-kube-api-access-5xx4s\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6\" (UID: \"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.583976 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.583935 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:26.733650 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.733622 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6"] Jun 02 04:53:26.735205 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:53:26.735173 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db595f9_d5bc_4d12_b9f8_a8fc4c56d51e.slice/crio-da7abc5cec28610829919c110bbfbad85f925e0a8ef2e9ea795797d2d8c8ba64 WatchSource:0}: Error finding container da7abc5cec28610829919c110bbfbad85f925e0a8ef2e9ea795797d2d8c8ba64: Status 404 returned error can't find the container with id da7abc5cec28610829919c110bbfbad85f925e0a8ef2e9ea795797d2d8c8ba64 Jun 02 04:53:26.968229 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.968189 2566 generic.go:358] "Generic (PLEG): container finished" podID="7f20b02e-46b5-4ebe-b501-a39eea266150" containerID="e8754b5c05fe9997b2397e6f27c8ff94fbe7aed81424d7cd0042b3ddd7f57ed3" exitCode=0 Jun 02 04:53:26.968696 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.968284 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" event={"ID":"7f20b02e-46b5-4ebe-b501-a39eea266150","Type":"ContainerDied","Data":"e8754b5c05fe9997b2397e6f27c8ff94fbe7aed81424d7cd0042b3ddd7f57ed3"} Jun 02 04:53:26.969860 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.969834 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" event={"ID":"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e","Type":"ContainerStarted","Data":"4ac63937e4614e395194b89fcdf18bdc3e523d683be4ae2b35aab3dee3019e56"} Jun 02 04:53:26.969978 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:26.969866 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" event={"ID":"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e","Type":"ContainerStarted","Data":"da7abc5cec28610829919c110bbfbad85f925e0a8ef2e9ea795797d2d8c8ba64"} Jun 02 04:53:28.982171 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:28.982120 2566 generic.go:358] "Generic (PLEG): container finished" podID="6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e" containerID="4ac63937e4614e395194b89fcdf18bdc3e523d683be4ae2b35aab3dee3019e56" exitCode=0 Jun 02 04:53:28.982578 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:28.982197 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" event={"ID":"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e","Type":"ContainerDied","Data":"4ac63937e4614e395194b89fcdf18bdc3e523d683be4ae2b35aab3dee3019e56"} Jun 02 04:53:28.983940 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:28.983917 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" event={"ID":"7f20b02e-46b5-4ebe-b501-a39eea266150","Type":"ContainerStarted","Data":"0c0d4364b7aac4b049c79a73dc28184b4337961c11e802bb966851a4d861a772"} Jun 02 04:53:29.020495 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:29.020430 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" podStartSLOduration=1.97579183 podStartE2EDuration="9.020413378s" podCreationTimestamp="2026-06-02 04:53:20 +0000 UTC" firstStartedPulling="2026-06-02 04:53:21.007962602 +0000 UTC m=+683.431732864" lastFinishedPulling="2026-06-02 04:53:28.052584155 +0000 UTC m=+690.476354412" observedRunningTime="2026-06-02 04:53:29.019928695 +0000 UTC m=+691.443698983" watchObservedRunningTime="2026-06-02 04:53:29.020413378 +0000 UTC m=+691.444183644" Jun 02 04:53:29.988816 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:29.988779 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" event={"ID":"6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e","Type":"ContainerStarted","Data":"2028561b904fb478c2c8823a6e0f3e21d7cb37fbad117d20dfe8213bf7b447d0"} Jun 02 04:53:30.032276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:30.032218 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" podStartSLOduration=3.868193186 podStartE2EDuration="4.032204406s" podCreationTimestamp="2026-06-02 04:53:26 +0000 UTC" firstStartedPulling="2026-06-02 04:53:28.982931156 +0000 UTC m=+691.406701401" lastFinishedPulling="2026-06-02 04:53:29.146942376 +0000 UTC m=+691.570712621" observedRunningTime="2026-06-02 04:53:30.030748274 +0000 UTC m=+692.454518539" watchObservedRunningTime="2026-06-02 04:53:30.032204406 +0000 UTC m=+692.455974696" Jun 02 04:53:30.849103 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:30.849063 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:30.849103 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:30.849110 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:30.861444 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:30.861410 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:31.004248 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:31.004215 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c" Jun 02 04:53:36.584786 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:36.584669 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:36.584786 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:36.584742 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:36.597481 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:36.597451 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:37.025429 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:37.025343 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6" Jun 02 04:53:41.547694 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.547647 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x"] Jun 02 04:53:41.562304 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.562272 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x"] Jun 02 04:53:41.562496 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.562424 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.564982 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.564957 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 02 04:53:41.641879 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.641838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.642073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.641890 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.642073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.641914 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.642073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.641954 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.642073 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.641977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nkqk\" (UniqueName: \"kubernetes.io/projected/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-kube-api-access-4nkqk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.642276 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.642064 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.742822 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.742777 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.742822 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.742830 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4nkqk\" (UniqueName: \"kubernetes.io/projected/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-kube-api-access-4nkqk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743087 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.742870 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743087 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.742895 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743087 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.742925 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743087 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.742945 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743337 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.743271 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743337 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.743286 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.743337 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.743330 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.745272 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.745239 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.745523 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.745507 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.751307 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.751281 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nkqk\" (UniqueName: \"kubernetes.io/projected/ff5455d1-4737-4c56-8ecb-857b0bdbabd8-kube-api-access-4nkqk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x\" (UID: \"ff5455d1-4737-4c56-8ecb-857b0bdbabd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:41.875951 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:41.875915 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:42.011878 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:42.011850 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x"] Jun 02 04:53:42.013638 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:53:42.013613 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5455d1_4737_4c56_8ecb_857b0bdbabd8.slice/crio-480a1a8d223b3725dd6857d947dbca7f25866220546b090865bb6aded0de2707 WatchSource:0}: Error finding container 480a1a8d223b3725dd6857d947dbca7f25866220546b090865bb6aded0de2707: Status 404 returned error can't find the container with id 480a1a8d223b3725dd6857d947dbca7f25866220546b090865bb6aded0de2707 Jun 02 04:53:42.033186 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:42.033153 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" event={"ID":"ff5455d1-4737-4c56-8ecb-857b0bdbabd8","Type":"ContainerStarted","Data":"480a1a8d223b3725dd6857d947dbca7f25866220546b090865bb6aded0de2707"} Jun 02 04:53:43.037944 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:43.037904 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" event={"ID":"ff5455d1-4737-4c56-8ecb-857b0bdbabd8","Type":"ContainerStarted","Data":"0c9d672ddd39b99aa8426356ae6efcf854a456abd16ab3289ae747711a2b9495"} Jun 02 04:53:44.043090 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:44.043051 2566 generic.go:358] "Generic (PLEG): container finished" podID="ff5455d1-4737-4c56-8ecb-857b0bdbabd8" containerID="0c9d672ddd39b99aa8426356ae6efcf854a456abd16ab3289ae747711a2b9495" exitCode=0 Jun 02 04:53:44.043510 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:44.043149 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" event={"ID":"ff5455d1-4737-4c56-8ecb-857b0bdbabd8","Type":"ContainerDied","Data":"0c9d672ddd39b99aa8426356ae6efcf854a456abd16ab3289ae747711a2b9495"} Jun 02 04:53:45.048768 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:45.048731 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" event={"ID":"ff5455d1-4737-4c56-8ecb-857b0bdbabd8","Type":"ContainerStarted","Data":"5bf65757fd7dabc55f41415db978b302bdd038d39a4627a56c3bac94ddb8e735"} Jun 02 04:53:45.075413 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:45.075357 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" podStartSLOduration=3.890497228 podStartE2EDuration="4.075342711s" podCreationTimestamp="2026-06-02 04:53:41 +0000 UTC" firstStartedPulling="2026-06-02 04:53:44.043702745 +0000 UTC m=+706.467472986" lastFinishedPulling="2026-06-02 04:53:44.228548225 +0000 UTC m=+706.652318469" observedRunningTime="2026-06-02 04:53:45.073312632 +0000 UTC m=+707.497082919" watchObservedRunningTime="2026-06-02 04:53:45.075342711 +0000 UTC m=+707.499113013" Jun 02 04:53:51.877042 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:51.876993 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:51.877494 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:51.877054 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:51.889615 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:51.889579 2566 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:53:52.086023 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:53:52.085990 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x" Jun 02 04:55:32.159123 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.159075 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55788849bd-hpxw6"] Jun 02 04:55:32.162867 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.162844 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.174083 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.174055 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55788849bd-hpxw6"] Jun 02 04:55:32.269103 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.269061 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-tls-cert\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.269337 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.269234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brfdg\" (UniqueName: \"kubernetes.io/projected/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-kube-api-access-brfdg\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.269337 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.269276 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-oidc-ca\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.370186 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.370122 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-brfdg\" (UniqueName: \"kubernetes.io/projected/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-kube-api-access-brfdg\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.370386 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.370194 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-oidc-ca\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.370386 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.370251 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-tls-cert\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.370945 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.370924 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-oidc-ca\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.372728 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.372710 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-tls-cert\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.378407 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.378388 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brfdg\" (UniqueName: \"kubernetes.io/projected/68c7320e-0cf6-449b-aa17-9cd5c1fbfb44-kube-api-access-brfdg\") pod \"authorino-55788849bd-hpxw6\" (UID: \"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44\") " pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.472113 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.472013 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55788849bd-hpxw6" Jun 02 04:55:32.617038 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:32.616997 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55788849bd-hpxw6"] Jun 02 04:55:32.618825 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:55:32.618789 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68c7320e_0cf6_449b_aa17_9cd5c1fbfb44.slice/crio-3fd093c7ccd05cec10cfdc8d540292ba4789def747f779418ac9d6ef795e046d WatchSource:0}: Error finding container 3fd093c7ccd05cec10cfdc8d540292ba4789def747f779418ac9d6ef795e046d: Status 404 returned error can't find the container with id 3fd093c7ccd05cec10cfdc8d540292ba4789def747f779418ac9d6ef795e046d Jun 02 04:55:33.464895 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.464857 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55788849bd-hpxw6" event={"ID":"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44","Type":"ContainerStarted","Data":"8258d6d9e4c8e1b4d8b756bd7c463926188c55b30d327e985876dbb79281d850"} Jun 02 04:55:33.464895 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.464900 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55788849bd-hpxw6" event={"ID":"68c7320e-0cf6-449b-aa17-9cd5c1fbfb44","Type":"ContainerStarted","Data":"3fd093c7ccd05cec10cfdc8d540292ba4789def747f779418ac9d6ef795e046d"} Jun 02 04:55:33.485574 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.485520 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-55788849bd-hpxw6" podStartSLOduration=1.046387394 podStartE2EDuration="1.48550551s" podCreationTimestamp="2026-06-02 04:55:32 +0000 UTC" firstStartedPulling="2026-06-02 04:55:32.620054767 +0000 UTC m=+815.043825009" lastFinishedPulling="2026-06-02 04:55:33.05917287 +0000 UTC m=+815.482943125" observedRunningTime="2026-06-02 04:55:33.483387235 +0000 UTC m=+815.907157498" watchObservedRunningTime="2026-06-02 04:55:33.48550551 +0000 UTC m=+815.909275772" Jun 02 04:55:33.513344 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.513296 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68cfdffbf-mx29n"] Jun 02 04:55:33.513604 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.513572 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-68cfdffbf-mx29n" podUID="5f9e8079-b30d-4511-9be1-69bb68cbdb22" containerName="authorino" containerID="cri-o://f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0" gracePeriod=30 Jun 02 04:55:33.777874 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.777847 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:55:33.892579 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.892540 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpp9f\" (UniqueName: \"kubernetes.io/projected/5f9e8079-b30d-4511-9be1-69bb68cbdb22-kube-api-access-gpp9f\") pod \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " Jun 02 04:55:33.892753 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.892602 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5f9e8079-b30d-4511-9be1-69bb68cbdb22-oidc-ca\") pod \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " Jun 02 04:55:33.892753 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.892711 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f9e8079-b30d-4511-9be1-69bb68cbdb22-tls-cert\") pod \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\" (UID: \"5f9e8079-b30d-4511-9be1-69bb68cbdb22\") " Jun 02 04:55:33.894738 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.894705 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f9e8079-b30d-4511-9be1-69bb68cbdb22-kube-api-access-gpp9f" (OuterVolumeSpecName: "kube-api-access-gpp9f") pod "5f9e8079-b30d-4511-9be1-69bb68cbdb22" (UID: "5f9e8079-b30d-4511-9be1-69bb68cbdb22"). InnerVolumeSpecName "kube-api-access-gpp9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:55:33.897558 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.897532 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9e8079-b30d-4511-9be1-69bb68cbdb22-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "5f9e8079-b30d-4511-9be1-69bb68cbdb22" (UID: "5f9e8079-b30d-4511-9be1-69bb68cbdb22"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 04:55:33.902928 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.902899 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f9e8079-b30d-4511-9be1-69bb68cbdb22-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5f9e8079-b30d-4511-9be1-69bb68cbdb22" (UID: "5f9e8079-b30d-4511-9be1-69bb68cbdb22"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 04:55:33.993698 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.993662 2566 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5f9e8079-b30d-4511-9be1-69bb68cbdb22-oidc-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:55:33.993698 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.993692 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f9e8079-b30d-4511-9be1-69bb68cbdb22-tls-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:55:33.993913 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:33.993704 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpp9f\" (UniqueName: \"kubernetes.io/projected/5f9e8079-b30d-4511-9be1-69bb68cbdb22-kube-api-access-gpp9f\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:55:34.470593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.470490 2566 generic.go:358] "Generic (PLEG): container finished" podID="5f9e8079-b30d-4511-9be1-69bb68cbdb22" containerID="f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0" exitCode=0 Jun 02 04:55:34.470593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.470558 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68cfdffbf-mx29n" Jun 02 04:55:34.470593 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.470576 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68cfdffbf-mx29n" event={"ID":"5f9e8079-b30d-4511-9be1-69bb68cbdb22","Type":"ContainerDied","Data":"f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0"} Jun 02 04:55:34.471156 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.470625 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68cfdffbf-mx29n" event={"ID":"5f9e8079-b30d-4511-9be1-69bb68cbdb22","Type":"ContainerDied","Data":"97860e91d145c33a38b2074fae7631f50076ae5851d8c51dd279dc91e8528349"} Jun 02 04:55:34.471156 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.470644 2566 scope.go:117] "RemoveContainer" containerID="f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0" Jun 02 04:55:34.479630 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.479609 2566 scope.go:117] "RemoveContainer" containerID="f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0" Jun 02 04:55:34.479937 ip-10-0-135-20 kubenswrapper[2566]: E0602 04:55:34.479918 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0\": container with ID starting with f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0 not found: ID does not exist" containerID="f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0" Jun 02 04:55:34.479984 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.479947 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0"} err="failed to get container status \"f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0\": rpc error: code = NotFound desc = could not find container \"f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0\": container with ID starting with f2a71788fc54df6f47ee22782f35d59821f4a06a5b7832fe6bbc51fce6309ba0 not found: ID does not exist" Jun 02 04:55:34.490665 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.490636 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68cfdffbf-mx29n"] Jun 02 04:55:34.494711 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:34.494687 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-68cfdffbf-mx29n"] Jun 02 04:55:36.120439 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:55:36.120401 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f9e8079-b30d-4511-9be1-69bb68cbdb22" path="/var/lib/kubelet/pods/5f9e8079-b30d-4511-9be1-69bb68cbdb22/volumes" Jun 02 04:56:58.111605 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:58.111533 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:56:58.114751 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:58.114729 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 04:56:59.655960 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:59.655919 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-bt9j9"] Jun 02 04:56:59.656384 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:59.656204 2566 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" podUID="faad3b49-e698-4e84-af1b-cf8863661376" containerName="manager" containerID="cri-o://e0d5c347d6d9d23cef955682192d0fa711daa7942647d211e5208879711bf422" gracePeriod=30 Jun 02 04:56:59.815647 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:59.815615 2566 generic.go:358] "Generic (PLEG): container finished" podID="faad3b49-e698-4e84-af1b-cf8863661376" containerID="e0d5c347d6d9d23cef955682192d0fa711daa7942647d211e5208879711bf422" exitCode=0 Jun 02 04:56:59.815819 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:59.815658 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" event={"ID":"faad3b49-e698-4e84-af1b-cf8863661376","Type":"ContainerDied","Data":"e0d5c347d6d9d23cef955682192d0fa711daa7942647d211e5208879711bf422"} Jun 02 04:56:59.908344 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:56:59.908276 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:57:00.076292 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.076254 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwks7\" (UniqueName: \"kubernetes.io/projected/faad3b49-e698-4e84-af1b-cf8863661376-kube-api-access-gwks7\") pod \"faad3b49-e698-4e84-af1b-cf8863661376\" (UID: \"faad3b49-e698-4e84-af1b-cf8863661376\") " Jun 02 04:57:00.078394 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.078359 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faad3b49-e698-4e84-af1b-cf8863661376-kube-api-access-gwks7" (OuterVolumeSpecName: "kube-api-access-gwks7") pod "faad3b49-e698-4e84-af1b-cf8863661376" (UID: "faad3b49-e698-4e84-af1b-cf8863661376"). InnerVolumeSpecName "kube-api-access-gwks7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 04:57:00.177376 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.177286 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwks7\" (UniqueName: \"kubernetes.io/projected/faad3b49-e698-4e84-af1b-cf8863661376-kube-api-access-gwks7\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 04:57:00.820752 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.820705 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" event={"ID":"faad3b49-e698-4e84-af1b-cf8863661376","Type":"ContainerDied","Data":"e2697d96cbd8874badd7f5571c999e8888a09927ff913e5a6a908ec5972a6260"} Jun 02 04:57:00.820752 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.820761 2566 scope.go:117] "RemoveContainer" containerID="e0d5c347d6d9d23cef955682192d0fa711daa7942647d211e5208879711bf422" Jun 02 04:57:00.821266 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.820729 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf7f788f6-bt9j9" Jun 02 04:57:00.843938 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.843907 2566 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-bt9j9"] Jun 02 04:57:00.849259 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:00.849227 2566 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-bt9j9"] Jun 02 04:57:01.684354 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684319 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-69zfv"] Jun 02 04:57:01.684736 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684724 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9e8079-b30d-4511-9be1-69bb68cbdb22" containerName="authorino" Jun 02 04:57:01.684831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684738 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9e8079-b30d-4511-9be1-69bb68cbdb22" containerName="authorino" Jun 02 04:57:01.684831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684751 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faad3b49-e698-4e84-af1b-cf8863661376" containerName="manager" Jun 02 04:57:01.684831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684756 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="faad3b49-e698-4e84-af1b-cf8863661376" containerName="manager" Jun 02 04:57:01.684831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684809 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="faad3b49-e698-4e84-af1b-cf8863661376" containerName="manager" Jun 02 04:57:01.684831 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.684817 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="5f9e8079-b30d-4511-9be1-69bb68cbdb22" containerName="authorino" Jun 02 04:57:01.689304 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.689287 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 04:57:01.692301 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.692280 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 04:57:01.692455 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.692281 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7m59l\"" Jun 02 04:57:01.700093 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.700058 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-69zfv"] Jun 02 04:57:01.792464 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.792429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-524gq\" (UniqueName: \"kubernetes.io/projected/564de0d3-8fba-4d7a-a18e-42f8d307b01f-kube-api-access-524gq\") pod \"maas-controller-7cf7f788f6-69zfv\" (UID: \"564de0d3-8fba-4d7a-a18e-42f8d307b01f\") " pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 04:57:01.893472 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.893440 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-524gq\" (UniqueName: \"kubernetes.io/projected/564de0d3-8fba-4d7a-a18e-42f8d307b01f-kube-api-access-524gq\") pod \"maas-controller-7cf7f788f6-69zfv\" (UID: \"564de0d3-8fba-4d7a-a18e-42f8d307b01f\") " pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 04:57:01.902100 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:01.902064 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-524gq\" (UniqueName: \"kubernetes.io/projected/564de0d3-8fba-4d7a-a18e-42f8d307b01f-kube-api-access-524gq\") pod \"maas-controller-7cf7f788f6-69zfv\" (UID: \"564de0d3-8fba-4d7a-a18e-42f8d307b01f\") " pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 04:57:02.000395 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.000299 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 04:57:02.119996 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.119965 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faad3b49-e698-4e84-af1b-cf8863661376" path="/var/lib/kubelet/pods/faad3b49-e698-4e84-af1b-cf8863661376/volumes" Jun 02 04:57:02.133171 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.133133 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cf7f788f6-69zfv"] Jun 02 04:57:02.135430 ip-10-0-135-20 kubenswrapper[2566]: W0602 04:57:02.135403 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564de0d3_8fba_4d7a_a18e_42f8d307b01f.slice/crio-dc554a3946c1cf237cfb6e6cc8182da32ea9a08f826c8f0407054cbdf286e67a WatchSource:0}: Error finding container dc554a3946c1cf237cfb6e6cc8182da32ea9a08f826c8f0407054cbdf286e67a: Status 404 returned error can't find the container with id dc554a3946c1cf237cfb6e6cc8182da32ea9a08f826c8f0407054cbdf286e67a Jun 02 04:57:02.830632 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.830595 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" event={"ID":"564de0d3-8fba-4d7a-a18e-42f8d307b01f","Type":"ContainerStarted","Data":"79ba33d69c5e7506269f6dc2c8f686fcbc10debd41b9a8199fdc72d84809c1d6"} Jun 02 04:57:02.830632 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.830632 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" event={"ID":"564de0d3-8fba-4d7a-a18e-42f8d307b01f","Type":"ContainerStarted","Data":"dc554a3946c1cf237cfb6e6cc8182da32ea9a08f826c8f0407054cbdf286e67a"} Jun 02 04:57:02.830849 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.830655 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 04:57:02.852199 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:02.852128 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" podStartSLOduration=1.4435659809999999 podStartE2EDuration="1.852112325s" podCreationTimestamp="2026-06-02 04:57:01 +0000 UTC" firstStartedPulling="2026-06-02 04:57:02.136727244 +0000 UTC m=+904.560497505" lastFinishedPulling="2026-06-02 04:57:02.545273608 +0000 UTC m=+904.969043849" observedRunningTime="2026-06-02 04:57:02.849637463 +0000 UTC m=+905.273407738" watchObservedRunningTime="2026-06-02 04:57:02.852112325 +0000 UTC m=+905.275882614" Jun 02 04:57:13.839407 ip-10-0-135-20 kubenswrapper[2566]: I0602 04:57:13.839373 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cf7f788f6-69zfv" Jun 02 05:00:00.148826 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.148738 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29672940-9psv4"] Jun 02 05:00:00.152104 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.152076 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:00.154563 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.154537 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4q6m\"" Jun 02 05:00:00.173776 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.173739 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672940-9psv4"] Jun 02 05:00:00.266665 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.266618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rchj\" (UniqueName: \"kubernetes.io/projected/99ff56d4-0812-4235-89ad-23cb13c2beb9-kube-api-access-9rchj\") pod \"maas-api-key-cleanup-29672940-9psv4\" (UID: \"99ff56d4-0812-4235-89ad-23cb13c2beb9\") " pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:00.367764 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.367721 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9rchj\" (UniqueName: \"kubernetes.io/projected/99ff56d4-0812-4235-89ad-23cb13c2beb9-kube-api-access-9rchj\") pod \"maas-api-key-cleanup-29672940-9psv4\" (UID: \"99ff56d4-0812-4235-89ad-23cb13c2beb9\") " pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:00.376969 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.376935 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rchj\" (UniqueName: \"kubernetes.io/projected/99ff56d4-0812-4235-89ad-23cb13c2beb9-kube-api-access-9rchj\") pod \"maas-api-key-cleanup-29672940-9psv4\" (UID: \"99ff56d4-0812-4235-89ad-23cb13c2beb9\") " pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:00.462469 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.462375 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:00.605021 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.604987 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672940-9psv4"] Jun 02 05:00:00.606126 ip-10-0-135-20 kubenswrapper[2566]: W0602 05:00:00.606101 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ff56d4_0812_4235_89ad_23cb13c2beb9.slice/crio-8725feee9a6545c0123e950adf6c1913707788611e04723888801abe362451c7 WatchSource:0}: Error finding container 8725feee9a6545c0123e950adf6c1913707788611e04723888801abe362451c7: Status 404 returned error can't find the container with id 8725feee9a6545c0123e950adf6c1913707788611e04723888801abe362451c7 Jun 02 05:00:00.607853 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:00.607834 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 05:00:01.555842 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:01.555800 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" event={"ID":"99ff56d4-0812-4235-89ad-23cb13c2beb9","Type":"ContainerStarted","Data":"8725feee9a6545c0123e950adf6c1913707788611e04723888801abe362451c7"} Jun 02 05:00:10.604077 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:10.604028 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" event={"ID":"99ff56d4-0812-4235-89ad-23cb13c2beb9","Type":"ContainerStarted","Data":"4ce7bc30d2c6bdab007dc846605d2d8136f16d56cf7384a571b1c9782b9a91fb"} Jun 02 05:00:10.623899 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:10.623835 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" podStartSLOduration=1.147783838 podStartE2EDuration="10.623815139s" podCreationTimestamp="2026-06-02 05:00:00 +0000 UTC" firstStartedPulling="2026-06-02 05:00:00.60796253 +0000 UTC m=+1083.031732770" lastFinishedPulling="2026-06-02 05:00:10.083993818 +0000 UTC m=+1092.507764071" observedRunningTime="2026-06-02 05:00:10.621748292 +0000 UTC m=+1093.045518554" watchObservedRunningTime="2026-06-02 05:00:10.623815139 +0000 UTC m=+1093.047585403" Jun 02 05:00:11.608869 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:11.608835 2566 generic.go:358] "Generic (PLEG): container finished" podID="99ff56d4-0812-4235-89ad-23cb13c2beb9" containerID="4ce7bc30d2c6bdab007dc846605d2d8136f16d56cf7384a571b1c9782b9a91fb" exitCode=0 Jun 02 05:00:11.609304 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:11.608929 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" event={"ID":"99ff56d4-0812-4235-89ad-23cb13c2beb9","Type":"ContainerDied","Data":"4ce7bc30d2c6bdab007dc846605d2d8136f16d56cf7384a571b1c9782b9a91fb"} Jun 02 05:00:12.740007 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:12.739980 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:12.778220 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:12.778174 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rchj\" (UniqueName: \"kubernetes.io/projected/99ff56d4-0812-4235-89ad-23cb13c2beb9-kube-api-access-9rchj\") pod \"99ff56d4-0812-4235-89ad-23cb13c2beb9\" (UID: \"99ff56d4-0812-4235-89ad-23cb13c2beb9\") " Jun 02 05:00:12.780296 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:12.780254 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99ff56d4-0812-4235-89ad-23cb13c2beb9-kube-api-access-9rchj" (OuterVolumeSpecName: "kube-api-access-9rchj") pod "99ff56d4-0812-4235-89ad-23cb13c2beb9" (UID: "99ff56d4-0812-4235-89ad-23cb13c2beb9"). InnerVolumeSpecName "kube-api-access-9rchj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:00:12.879747 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:12.879647 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rchj\" (UniqueName: \"kubernetes.io/projected/99ff56d4-0812-4235-89ad-23cb13c2beb9-kube-api-access-9rchj\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 05:00:13.618065 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:13.618026 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" Jun 02 05:00:13.618237 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:13.618029 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-9psv4" event={"ID":"99ff56d4-0812-4235-89ad-23cb13c2beb9","Type":"ContainerDied","Data":"8725feee9a6545c0123e950adf6c1913707788611e04723888801abe362451c7"} Jun 02 05:00:13.618237 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:00:13.618161 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8725feee9a6545c0123e950adf6c1913707788611e04723888801abe362451c7" Jun 02 05:01:58.144277 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:01:58.144248 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:01:58.148668 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:01:58.148643 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:06:58.179287 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:06:58.179259 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:06:58.185264 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:06:58.185244 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:11:58.216560 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:11:58.216528 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:11:58.223530 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:11:58.223508 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:15:00.140787 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.140700 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29672955-7fgxl"] Jun 02 05:15:00.143265 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.141071 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ff56d4-0812-4235-89ad-23cb13c2beb9" containerName="cleanup" Jun 02 05:15:00.143265 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.141082 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ff56d4-0812-4235-89ad-23cb13c2beb9" containerName="cleanup" Jun 02 05:15:00.143265 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.141164 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="99ff56d4-0812-4235-89ad-23cb13c2beb9" containerName="cleanup" Jun 02 05:15:00.144245 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.144227 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:15:00.146529 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.146506 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r4q6m\"" Jun 02 05:15:00.158324 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.158286 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672955-7fgxl"] Jun 02 05:15:00.205710 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.205670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crg2l\" (UniqueName: \"kubernetes.io/projected/5e2d2085-97d5-4b51-909d-bbb017e6f1aa-kube-api-access-crg2l\") pod \"maas-api-key-cleanup-29672955-7fgxl\" (UID: \"5e2d2085-97d5-4b51-909d-bbb017e6f1aa\") " pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:15:00.307275 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.307232 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-crg2l\" (UniqueName: \"kubernetes.io/projected/5e2d2085-97d5-4b51-909d-bbb017e6f1aa-kube-api-access-crg2l\") pod \"maas-api-key-cleanup-29672955-7fgxl\" (UID: \"5e2d2085-97d5-4b51-909d-bbb017e6f1aa\") " pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:15:00.316333 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.316300 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crg2l\" (UniqueName: \"kubernetes.io/projected/5e2d2085-97d5-4b51-909d-bbb017e6f1aa-kube-api-access-crg2l\") pod \"maas-api-key-cleanup-29672955-7fgxl\" (UID: \"5e2d2085-97d5-4b51-909d-bbb017e6f1aa\") " pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:15:00.455346 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.455228 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:15:00.594963 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.594931 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672955-7fgxl"] Jun 02 05:15:00.596351 ip-10-0-135-20 kubenswrapper[2566]: W0602 05:15:00.596322 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2d2085_97d5_4b51_909d_bbb017e6f1aa.slice/crio-9c07f72207d36d05fc2ab2f046ff6cd4ee61fa972f5dd469cb44f8bb55304d1f WatchSource:0}: Error finding container 9c07f72207d36d05fc2ab2f046ff6cd4ee61fa972f5dd469cb44f8bb55304d1f: Status 404 returned error can't find the container with id 9c07f72207d36d05fc2ab2f046ff6cd4ee61fa972f5dd469cb44f8bb55304d1f Jun 02 05:15:00.598042 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:00.598024 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 05:15:01.154337 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:01.154236 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" event={"ID":"5e2d2085-97d5-4b51-909d-bbb017e6f1aa","Type":"ContainerStarted","Data":"cdc48aa474b6b7ab0a554edb87db05a97954d1afc00f09914fc131394049c3f4"} Jun 02 05:15:01.154337 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:01.154278 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" event={"ID":"5e2d2085-97d5-4b51-909d-bbb017e6f1aa","Type":"ContainerStarted","Data":"9c07f72207d36d05fc2ab2f046ff6cd4ee61fa972f5dd469cb44f8bb55304d1f"} Jun 02 05:15:01.173404 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:01.173350 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" podStartSLOduration=1.173334854 podStartE2EDuration="1.173334854s" podCreationTimestamp="2026-06-02 05:15:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:15:01.170198109 +0000 UTC m=+1983.593968372" watchObservedRunningTime="2026-06-02 05:15:01.173334854 +0000 UTC m=+1983.597105117" Jun 02 05:15:02.159921 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:02.159885 2566 generic.go:358] "Generic (PLEG): container finished" podID="5e2d2085-97d5-4b51-909d-bbb017e6f1aa" containerID="cdc48aa474b6b7ab0a554edb87db05a97954d1afc00f09914fc131394049c3f4" exitCode=0 Jun 02 05:15:02.160373 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:02.159977 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" event={"ID":"5e2d2085-97d5-4b51-909d-bbb017e6f1aa","Type":"ContainerDied","Data":"cdc48aa474b6b7ab0a554edb87db05a97954d1afc00f09914fc131394049c3f4"} Jun 02 05:15:03.295257 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:03.295232 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:15:03.436643 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:03.436542 2566 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crg2l\" (UniqueName: \"kubernetes.io/projected/5e2d2085-97d5-4b51-909d-bbb017e6f1aa-kube-api-access-crg2l\") pod \"5e2d2085-97d5-4b51-909d-bbb017e6f1aa\" (UID: \"5e2d2085-97d5-4b51-909d-bbb017e6f1aa\") " Jun 02 05:15:03.438609 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:03.438586 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2d2085-97d5-4b51-909d-bbb017e6f1aa-kube-api-access-crg2l" (OuterVolumeSpecName: "kube-api-access-crg2l") pod "5e2d2085-97d5-4b51-909d-bbb017e6f1aa" (UID: "5e2d2085-97d5-4b51-909d-bbb017e6f1aa"). InnerVolumeSpecName "kube-api-access-crg2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 05:15:03.537245 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:03.537204 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crg2l\" (UniqueName: \"kubernetes.io/projected/5e2d2085-97d5-4b51-909d-bbb017e6f1aa-kube-api-access-crg2l\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\"" Jun 02 05:15:04.168398 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:04.168357 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" event={"ID":"5e2d2085-97d5-4b51-909d-bbb017e6f1aa","Type":"ContainerDied","Data":"9c07f72207d36d05fc2ab2f046ff6cd4ee61fa972f5dd469cb44f8bb55304d1f"} Jun 02 05:15:04.168398 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:04.168402 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c07f72207d36d05fc2ab2f046ff6cd4ee61fa972f5dd469cb44f8bb55304d1f" Jun 02 05:15:04.168604 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:15:04.168436 2566 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-7fgxl" Jun 02 05:16:58.252281 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:16:58.252252 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:16:58.259793 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:16:58.259770 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:18:28.993712 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:28.993676 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55788849bd-hpxw6_68c7320e-0cf6-449b-aa17-9cd5c1fbfb44/authorino/0.log" Jun 02 05:18:33.959457 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:33.959423 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672940-9psv4_99ff56d4-0812-4235-89ad-23cb13c2beb9/cleanup/0.log" Jun 02 05:18:34.069426 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.069394 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672955-7fgxl_5e2d2085-97d5-4b51-909d-bbb017e6f1aa/cleanup/0.log" Jun 02 05:18:34.185883 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.185845 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7cf7f788f6-69zfv_564de0d3-8fba-4d7a-a18e-42f8d307b01f/manager/0.log" Jun 02 05:18:34.294159 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.294060 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-sjb96_bcb9f931-ede4-494d-9217-4b60393e563e/server/0.log" Jun 02 05:18:34.407510 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.407470 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-89j4n_30b6ba1a-de1a-4563-8c9c-55314dfd2977/manager/0.log" Jun 02 05:18:34.646936 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.646906 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5b979d7-bpzjr_861a43e3-0d05-4eb0-bd13-27d788a85c6d/copy-manifests/0.log" Jun 02 05:18:34.654003 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.653972 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5b979d7-bpzjr_861a43e3-0d05-4eb0-bd13-27d788a85c6d/manager/0.log" Jun 02 05:18:34.878520 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:34.878493 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-nhjp9_f00d7938-156b-405e-ac52-beae953ccc0e/postgres/0.log" Jun 02 05:18:35.614708 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.614676 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6_20c8f841-3813-43cd-adc2-00d2a79dacb4/util/0.log" Jun 02 05:18:35.623367 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.623336 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6_20c8f841-3813-43cd-adc2-00d2a79dacb4/pull/0.log" Jun 02 05:18:35.629205 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.629177 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6_20c8f841-3813-43cd-adc2-00d2a79dacb4/extract/0.log" Jun 02 05:18:35.738099 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.738067 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47_34c85b26-2bb3-4e9c-83a4-d73410512f41/util/0.log" Jun 02 05:18:35.745254 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.745233 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47_34c85b26-2bb3-4e9c-83a4-d73410512f41/pull/0.log" Jun 02 05:18:35.751953 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.751936 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47_34c85b26-2bb3-4e9c-83a4-d73410512f41/extract/0.log" Jun 02 05:18:35.860720 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.860688 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz_cb362b3b-24d4-415c-a209-48a9884ec986/util/0.log" Jun 02 05:18:35.866609 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.866551 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz_cb362b3b-24d4-415c-a209-48a9884ec986/pull/0.log" Jun 02 05:18:35.872334 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.872315 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz_cb362b3b-24d4-415c-a209-48a9884ec986/extract/0.log" Jun 02 05:18:35.974322 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.974294 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8_0082c667-16fd-4340-b96b-18e6b73b4f7f/extract/0.log" Jun 02 05:18:35.979998 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.979973 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8_0082c667-16fd-4340-b96b-18e6b73b4f7f/util/0.log" Jun 02 05:18:35.985644 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:35.985619 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8_0082c667-16fd-4340-b96b-18e6b73b4f7f/pull/0.log" Jun 02 05:18:36.100042 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:36.100008 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55788849bd-hpxw6_68c7320e-0cf6-449b-aa17-9cd5c1fbfb44/authorino/0.log" Jun 02 05:18:36.429969 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:36.429942 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-z4hdd_37ee2087-7e53-43f6-b277-6ff3328240a4/kuadrant-console-plugin/0.log" Jun 02 05:18:37.587797 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:37.587766 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65977b746f-vdjwv_8f4cddb8-a86e-41f7-b747-8e6be4887e66/kube-auth-proxy/0.log" Jun 02 05:18:38.264984 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:38.264944 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c_7f20b02e-46b5-4ebe-b501-a39eea266150/storage-initializer/0.log" Jun 02 05:18:38.272053 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:38.272020 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-67n8c_7f20b02e-46b5-4ebe-b501-a39eea266150/main/0.log" Jun 02 05:18:38.408005 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:38.407970 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6_6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e/storage-initializer/0.log" Jun 02 05:18:38.427875 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:38.427847 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-7z9r6_6db595f9-d5bc-4d12-b9f8-a8fc4c56d51e/main/0.log" Jun 02 05:18:38.666371 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:38.666339 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x_ff5455d1-4737-4c56-8ecb-857b0bdbabd8/storage-initializer/0.log" Jun 02 05:18:38.673574 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:38.673552 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9ddx4x_ff5455d1-4737-4c56-8ecb-857b0bdbabd8/main/0.log" Jun 02 05:18:45.871589 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:45.871553 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wc5md_ad29c16f-6b87-412b-b764-b39c6f516012/global-pull-secret-syncer/0.log" Jun 02 05:18:45.981956 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:45.981928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zpqvp_8776f2db-ee73-4f3d-a9e3-4a339c6e51a9/konnectivity-agent/0.log" Jun 02 05:18:46.004519 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:46.004487 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-20.ec2.internal_12dc1b6bf110d069bf03b7ee9f029d82/haproxy/0.log" Jun 02 05:18:50.265344 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.265310 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6_20c8f841-3813-43cd-adc2-00d2a79dacb4/extract/0.log" Jun 02 05:18:50.289893 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.289860 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6_20c8f841-3813-43cd-adc2-00d2a79dacb4/util/0.log" Jun 02 05:18:50.313938 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.313907 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rf6j6_20c8f841-3813-43cd-adc2-00d2a79dacb4/pull/0.log" Jun 02 05:18:50.343069 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.343041 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47_34c85b26-2bb3-4e9c-83a4-d73410512f41/extract/0.log" Jun 02 05:18:50.366682 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.366652 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47_34c85b26-2bb3-4e9c-83a4-d73410512f41/util/0.log" Jun 02 05:18:50.392512 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.392459 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bjh47_34c85b26-2bb3-4e9c-83a4-d73410512f41/pull/0.log" Jun 02 05:18:50.422426 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.422378 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz_cb362b3b-24d4-415c-a209-48a9884ec986/extract/0.log" Jun 02 05:18:50.448718 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.448691 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz_cb362b3b-24d4-415c-a209-48a9884ec986/util/0.log" Jun 02 05:18:50.480392 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.480359 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zsqxz_cb362b3b-24d4-415c-a209-48a9884ec986/pull/0.log" Jun 02 05:18:50.514036 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.514009 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8_0082c667-16fd-4340-b96b-18e6b73b4f7f/extract/0.log" Jun 02 05:18:50.542612 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.542536 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8_0082c667-16fd-4340-b96b-18e6b73b4f7f/util/0.log" Jun 02 05:18:50.570276 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.570246 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w4gr8_0082c667-16fd-4340-b96b-18e6b73b4f7f/pull/0.log" Jun 02 05:18:50.803672 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.803578 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55788849bd-hpxw6_68c7320e-0cf6-449b-aa17-9cd5c1fbfb44/authorino/0.log" Jun 02 05:18:50.911732 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:50.911704 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-z4hdd_37ee2087-7e53-43f6-b277-6ff3328240a4/kuadrant-console-plugin/0.log" Jun 02 05:18:52.642691 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.642660 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/alertmanager/0.log" Jun 02 05:18:52.674745 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.674718 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/config-reloader/0.log" Jun 02 05:18:52.701444 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.701411 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/kube-rbac-proxy-web/0.log" Jun 02 05:18:52.724471 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.724444 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/kube-rbac-proxy/0.log" Jun 02 05:18:52.750315 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.750288 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/kube-rbac-proxy-metric/0.log" Jun 02 05:18:52.777931 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.777901 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/prom-label-proxy/0.log" Jun 02 05:18:52.813188 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:52.813155 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4ce50496-003d-4540-9dc3-f4182a66fef9/init-config-reloader/0.log" Jun 02 05:18:53.052635 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.052603 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-t6785_e5c2750a-a1f1-4747-8781-95d65cc299f8/monitoring-plugin/0.log" Jun 02 05:18:53.274406 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.274324 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-phw6w_ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42/node-exporter/0.log" Jun 02 05:18:53.302920 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.302895 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-phw6w_ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42/kube-rbac-proxy/0.log" Jun 02 05:18:53.324839 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.324809 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-phw6w_ba2924c0-d4d0-4dbb-9b16-e9bd9daaad42/init-textfile/0.log" Jun 02 05:18:53.741026 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.740997 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78f698bbc-62kfd_f8571773-8d33-445c-9783-dab4aa2eaee4/telemeter-client/0.log" Jun 02 05:18:53.764261 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.764237 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78f698bbc-62kfd_f8571773-8d33-445c-9783-dab4aa2eaee4/reload/0.log" Jun 02 05:18:53.787026 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:53.787003 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78f698bbc-62kfd_f8571773-8d33-445c-9783-dab4aa2eaee4/kube-rbac-proxy/0.log" Jun 02 05:18:56.136525 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.136496 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-764cf7446c-gzrzr_e1473022-da19-4ce4-aee9-1b46c86a8ded/console/0.log" Jun 02 05:18:56.665153 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.665112 2566 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4"] Jun 02 05:18:56.665684 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.665662 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2d2085-97d5-4b51-909d-bbb017e6f1aa" containerName="cleanup" Jun 02 05:18:56.665684 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.665684 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2d2085-97d5-4b51-909d-bbb017e6f1aa" containerName="cleanup" Jun 02 05:18:56.665877 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.665806 2566 memory_manager.go:356] "RemoveStaleState removing state" podUID="5e2d2085-97d5-4b51-909d-bbb017e6f1aa" containerName="cleanup" Jun 02 05:18:56.669049 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.669029 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.671738 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.671716 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-htn26\"/\"openshift-service-ca.crt\"" Jun 02 05:18:56.671738 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.671737 2566 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-htn26\"/\"kube-root-ca.crt\"" Jun 02 05:18:56.672237 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.672219 2566 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-htn26\"/\"default-dockercfg-c9nmg\"" Jun 02 05:18:56.679081 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.679052 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4"] Jun 02 05:18:56.775455 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.775418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-proc\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.775455 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.775459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-lib-modules\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.775733 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.775555 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-podres\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.775733 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.775618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjdd6\" (UniqueName: \"kubernetes.io/projected/79efabc9-f974-48ef-8d97-69b5d7de6db3-kube-api-access-cjdd6\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.775733 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.775675 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-sys\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876778 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876739 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-sys\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876810 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-proc\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876831 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-lib-modules\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876863 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-podres\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876889 2566 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cjdd6\" (UniqueName: \"kubernetes.io/projected/79efabc9-f974-48ef-8d97-69b5d7de6db3-kube-api-access-cjdd6\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876886 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-sys\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876937 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-proc\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.876989 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876983 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-podres\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.877296 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.876988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/79efabc9-f974-48ef-8d97-69b5d7de6db3-lib-modules\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.884873 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.884851 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjdd6\" (UniqueName: \"kubernetes.io/projected/79efabc9-f974-48ef-8d97-69b5d7de6db3-kube-api-access-cjdd6\") pod \"perf-node-gather-daemonset-tj8z4\" (UID: \"79efabc9-f974-48ef-8d97-69b5d7de6db3\") " pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:56.980865 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:56.980778 2566 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:57.120387 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:57.120362 2566 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4"] Jun 02 05:18:57.121945 ip-10-0-135-20 kubenswrapper[2566]: W0602 05:18:57.121913 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod79efabc9_f974_48ef_8d97_69b5d7de6db3.slice/crio-b1cce30965f6ee87f5566a7eb5812061431a85f2f61853acb8f20a69676f4d71 WatchSource:0}: Error finding container b1cce30965f6ee87f5566a7eb5812061431a85f2f61853acb8f20a69676f4d71: Status 404 returned error can't find the container with id b1cce30965f6ee87f5566a7eb5812061431a85f2f61853acb8f20a69676f4d71 Jun 02 05:18:57.674447 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:57.674422 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sjg59_8a5ad758-ecd6-46a7-88d2-d17354c1b7d5/dns/0.log" Jun 02 05:18:57.695769 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:57.695739 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sjg59_8a5ad758-ecd6-46a7-88d2-d17354c1b7d5/kube-rbac-proxy/0.log" Jun 02 05:18:57.755927 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:57.755897 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4v4x_d85df90f-0704-466b-bfc3-be341c63a82a/dns-node-resolver/0.log" Jun 02 05:18:58.105846 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:58.105804 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" event={"ID":"79efabc9-f974-48ef-8d97-69b5d7de6db3","Type":"ContainerStarted","Data":"84d8522007155141030e765d2f858c63dd25c22fc5624f7160c014699340a9c7"} Jun 02 05:18:58.105846 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:58.105849 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:18:58.106219 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:58.105866 2566 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" event={"ID":"79efabc9-f974-48ef-8d97-69b5d7de6db3","Type":"ContainerStarted","Data":"b1cce30965f6ee87f5566a7eb5812061431a85f2f61853acb8f20a69676f4d71"} Jun 02 05:18:58.124520 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:58.124462 2566 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" podStartSLOduration=2.124444616 podStartE2EDuration="2.124444616s" podCreationTimestamp="2026-06-02 05:18:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:18:58.121704601 +0000 UTC m=+2220.545474856" watchObservedRunningTime="2026-06-02 05:18:58.124444616 +0000 UTC m=+2220.548214879" Jun 02 05:18:58.280334 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:58.280306 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66dc564ff9-6fcj8_34b27fbd-93e6-432d-9fac-6a7cb7642226/registry/0.log" Jun 02 05:18:58.325974 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:58.325945 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ndsfj_34e12dd2-625a-412d-b26c-206d4f40b16e/node-ca/0.log" Jun 02 05:18:59.366445 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:18:59.366413 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65977b746f-vdjwv_8f4cddb8-a86e-41f7-b747-8e6be4887e66/kube-auth-proxy/0.log" Jun 02 05:19:00.111333 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:00.111304 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-czq5w_16f61d36-7264-4685-811e-9bdfa53612e8/serve-healthcheck-canary/0.log" Jun 02 05:19:00.806240 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:00.806208 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qdb2w_d58f9f19-c5d4-41a2-a6b9-9421e39aeca4/kube-rbac-proxy/0.log" Jun 02 05:19:00.827938 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:00.827904 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qdb2w_d58f9f19-c5d4-41a2-a6b9-9421e39aeca4/exporter/0.log" Jun 02 05:19:00.858560 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:00.858528 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qdb2w_d58f9f19-c5d4-41a2-a6b9-9421e39aeca4/extractor/0.log" Jun 02 05:19:03.113474 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.113442 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672940-9psv4_99ff56d4-0812-4235-89ad-23cb13c2beb9/cleanup/0.log" Jun 02 05:19:03.138954 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.138926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672955-7fgxl_5e2d2085-97d5-4b51-909d-bbb017e6f1aa/cleanup/0.log" Jun 02 05:19:03.208891 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.208857 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7cf7f788f6-69zfv_564de0d3-8fba-4d7a-a18e-42f8d307b01f/manager/0.log" Jun 02 05:19:03.233309 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.233282 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-sjb96_bcb9f931-ede4-494d-9217-4b60393e563e/server/0.log" Jun 02 05:19:03.280932 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.280900 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-89j4n_30b6ba1a-de1a-4563-8c9c-55314dfd2977/manager/0.log" Jun 02 05:19:03.451094 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.451009 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5b979d7-bpzjr_861a43e3-0d05-4eb0-bd13-27d788a85c6d/manager/0.log" Jun 02 05:19:03.480960 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.480928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5b979d7-bpzjr_861a43e3-0d05-4eb0-bd13-27d788a85c6d/copy-manifests/0.log" Jun 02 05:19:03.561457 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:03.561428 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-nhjp9_f00d7938-156b-405e-ac52-beae953ccc0e/postgres/0.log" Jun 02 05:19:04.120733 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:04.120707 2566 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-htn26/perf-node-gather-daemonset-tj8z4" Jun 02 05:19:04.946275 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:04.946239 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f8df7c74b-c9ngb_a1764b75-cf72-4747-b57f-04f9a4b75e88/manager/0.log" Jun 02 05:19:04.974246 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:04.974216 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-nbrkv_cdd319e3-1f35-4b03-b8bd-ccdf327897dc/openshift-lws-operator/0.log" Jun 02 05:19:10.276481 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:10.276447 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-5rkhl_e4c2faa7-ddc4-407f-806f-071afd1ef8c7/migrator/0.log" Jun 02 05:19:10.301759 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:10.301727 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-5rkhl_e4c2faa7-ddc4-407f-806f-071afd1ef8c7/graceful-termination/0.log" Jun 02 05:19:11.836116 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.836079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4gjpt_8d48b463-ab69-454c-b60c-9251bd561dd7/kube-multus/0.log" Jun 02 05:19:11.860396 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.860358 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/kube-multus-additional-cni-plugins/0.log" Jun 02 05:19:11.884759 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.884730 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/egress-router-binary-copy/0.log" Jun 02 05:19:11.908479 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.908453 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/cni-plugins/0.log" Jun 02 05:19:11.932324 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.932295 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/bond-cni-plugin/0.log" Jun 02 05:19:11.954204 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.954170 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/routeoverride-cni/0.log" Jun 02 05:19:11.976997 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:11.976969 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/whereabouts-cni-bincopy/0.log" Jun 02 05:19:12.000333 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:12.000304 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6gtwr_395e222e-8be8-4bc2-aaeb-92ab5148229d/whereabouts-cni/0.log" Jun 02 05:19:12.547584 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:12.547104 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l8f45_b05f924c-4cf3-4db7-bb80-baa2e515793e/network-metrics-daemon/0.log" Jun 02 05:19:12.611058 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:12.611024 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l8f45_b05f924c-4cf3-4db7-bb80-baa2e515793e/kube-rbac-proxy/0.log" Jun 02 05:19:13.868698 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.868665 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-controller/0.log" Jun 02 05:19:13.887314 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.887281 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/0.log" Jun 02 05:19:13.897118 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.897089 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/ovn-acl-logging/1.log" Jun 02 05:19:13.915927 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.915899 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/kube-rbac-proxy-node/0.log" Jun 02 05:19:13.939911 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.939881 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 05:19:13.965806 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.965777 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/northd/0.log" Jun 02 05:19:13.990120 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:13.990079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/nbdb/0.log" Jun 02 05:19:14.012834 ip-10-0-135-20 kubenswrapper[2566]: I0602 05:19:14.012803 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hjcp9_6d79be2a-3c4d-4eef-8793-27c3d4718282/sbdb/0.log"