Jun 06 22:03:00.148372 ip-10-0-136-77 systemd[1]: Starting Kubernetes Kubelet... Jun 06 22:03:00.491427 ip-10-0-136-77 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 06 22:03:00.491427 ip-10-0-136-77 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 06 22:03:00.491427 ip-10-0-136-77 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 06 22:03:00.491427 ip-10-0-136-77 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 06 22:03:00.491427 ip-10-0-136-77 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 06 22:03:00.492290 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.492207 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 06 22:03:00.498214 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498198 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:03:00.498214 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498214 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498217 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498221 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498224 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498226 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498230 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498233 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498235 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498238 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498241 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498243 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498246 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498250 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498253 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498256 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498258 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498261 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498265 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498269 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:03:00.498280 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498272 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498275 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498278 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498281 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498284 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498287 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498290 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498293 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498295 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498298 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498302 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498306 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498309 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498312 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498315 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498317 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498321 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498324 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498327 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498329 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:03:00.498929 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498371 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498428 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498433 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498437 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498442 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498446 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498450 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498454 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498459 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498464 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498468 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498472 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498477 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498481 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498491 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498496 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.498500 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499217 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499240 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499246 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:03:00.499631 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499254 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499258 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499704 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499842 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499849 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499855 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499858 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499862 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499865 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499868 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499871 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499874 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499877 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499880 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499883 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499886 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499888 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499891 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499894 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:03:00.500166 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499898 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499901 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499903 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499906 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499909 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499911 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499914 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499916 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499919 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499922 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499925 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499927 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499930 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499932 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499935 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499937 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499941 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499944 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499947 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:03:00.500611 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.499950 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500712 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500720 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500724 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500727 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500730 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500733 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500737 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500741 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500744 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500747 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500749 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500753 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500758 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500761 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500764 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500766 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500769 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500772 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:03:00.501059 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500788 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500792 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500795 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500798 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500801 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500804 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500807 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500809 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500812 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500814 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500817 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500820 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500822 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500826 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500830 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500833 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500836 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500839 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500842 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:03:00.501542 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500844 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500847 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500849 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500852 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500855 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500857 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500860 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500862 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500865 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500867 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500870 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500872 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500874 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500878 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500880 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500883 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500886 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500889 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500891 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500894 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:03:00.502003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500897 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500899 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500902 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500904 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500907 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500909 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500912 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500915 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500919 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500922 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500924 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500927 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500929 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500932 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500934 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500937 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500940 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500942 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500945 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500947 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:03:00.502507 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500949 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500952 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500954 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500957 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500959 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500962 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500964 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500967 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500969 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500972 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500974 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500976 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500980 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500983 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500985 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500988 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500990 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500993 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500995 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:03:00.502985 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.500999 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501001 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501004 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501406 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501412 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501415 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501417 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501420 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501422 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501425 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501427 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501430 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501433 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501436 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501438 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501441 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501443 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501446 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501450 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:03:00.503487 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501453 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501457 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501459 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501462 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501465 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501468 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501471 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501475 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501478 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501481 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501484 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501488 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501492 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501495 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501498 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501501 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501504 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501507 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501510 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501512 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:03:00.503972 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501515 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501517 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501520 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501522 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501525 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501527 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501529 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501532 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501534 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501537 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501539 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501542 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501545 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501548 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501551 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501554 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501556 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501559 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501562 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501564 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:03:00.504472 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501568 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501570 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501573 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501576 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501578 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501581 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501583 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501586 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501588 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501591 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501593 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501596 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501598 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501601 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501603 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501605 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501608 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501612 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501615 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:03:00.504949 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501617 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501619 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501622 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501624 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501626 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501629 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501631 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501635 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501637 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501640 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501643 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501645 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501647 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501650 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501653 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501656 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501659 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501661 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501664 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501666 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:03:00.505448 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501669 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501671 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501674 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.501676 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501748 2559 flags.go:64] FLAG: --address="0.0.0.0" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501758 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501765 2559 flags.go:64] FLAG: --anonymous-auth="true" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501770 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501783 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501786 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501791 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501796 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501799 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501801 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501805 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501808 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501811 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501814 2559 flags.go:64] FLAG: --cgroup-root="" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501817 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501820 2559 flags.go:64] FLAG: --client-ca-file="" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501822 2559 flags.go:64] FLAG: --cloud-provider="external" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501826 2559 flags.go:64] FLAG: --cluster-dns="[]" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501830 2559 flags.go:64] FLAG: --cluster-domain="" Jun 06 22:03:00.505939 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501833 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501836 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501838 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501842 2559 flags.go:64] FLAG: --container-log-max-files="5" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501851 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501854 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501857 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501860 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501863 2559 flags.go:64] FLAG: --contention-profiling="false" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501866 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501869 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501872 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501875 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501879 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501882 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501885 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501887 2559 flags.go:64] FLAG: --enable-load-reader="false" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501890 2559 flags.go:64] FLAG: --enable-server="true" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501893 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501898 2559 flags.go:64] FLAG: --event-burst="100" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501901 2559 flags.go:64] FLAG: --event-qps="50" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501903 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501906 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501909 2559 flags.go:64] FLAG: --eviction-hard="" Jun 06 22:03:00.506551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501913 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501915 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501918 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501921 2559 flags.go:64] FLAG: --eviction-soft="" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501924 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501927 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501930 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501933 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501936 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501939 2559 flags.go:64] FLAG: --fail-swap-on="true" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501942 2559 flags.go:64] FLAG: --feature-gates="" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501945 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501949 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501953 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501956 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501959 2559 flags.go:64] FLAG: --healthz-port="10248" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501962 2559 flags.go:64] FLAG: --help="false" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501964 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501968 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501970 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501973 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501977 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501980 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501983 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 06 22:03:00.507142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501985 2559 flags.go:64] FLAG: --image-service-endpoint="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501988 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501991 2559 flags.go:64] FLAG: --kube-api-burst="100" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501994 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501997 2559 flags.go:64] FLAG: --kube-api-qps="50" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.501999 2559 flags.go:64] FLAG: --kube-reserved="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502002 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502005 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502009 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502011 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502014 2559 flags.go:64] FLAG: --lock-file="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502017 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502020 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502022 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502028 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502030 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502033 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502036 2559 flags.go:64] FLAG: --logging-format="text" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502039 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502042 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502046 2559 flags.go:64] FLAG: --manifest-url="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502048 2559 flags.go:64] FLAG: --manifest-url-header="" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502053 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502056 2559 flags.go:64] FLAG: --max-open-files="1000000" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502060 2559 flags.go:64] FLAG: --max-pods="110" Jun 06 22:03:00.507752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502063 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502066 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502069 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502071 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502074 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502077 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502080 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502087 2559 flags.go:64] FLAG: --node-status-max-images="50" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502090 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502107 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502110 2559 flags.go:64] FLAG: --pod-cidr="" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502113 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502118 2559 flags.go:64] FLAG: --pod-manifest-path="" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502121 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502124 2559 flags.go:64] FLAG: --pods-per-core="0" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502127 2559 flags.go:64] FLAG: --port="10250" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502130 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502133 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ce5b9c56a3a86c82" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502136 2559 flags.go:64] FLAG: --qos-reserved="" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502139 2559 flags.go:64] FLAG: --read-only-port="10255" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502142 2559 flags.go:64] FLAG: --register-node="true" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502144 2559 flags.go:64] FLAG: --register-with-taints="" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502148 2559 flags.go:64] FLAG: --registry-burst="10" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502150 2559 flags.go:64] FLAG: --registry-qps="5" Jun 06 22:03:00.508441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502153 2559 flags.go:64] FLAG: --reserved-cpus="" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502156 2559 flags.go:64] FLAG: --reserved-memory="" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502160 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502163 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502167 2559 flags.go:64] FLAG: --rotate-certificates="false" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502170 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502174 2559 flags.go:64] FLAG: --runonce="false" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502177 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502180 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502183 2559 flags.go:64] FLAG: --seccomp-default="false" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502185 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502188 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502191 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502194 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502197 2559 flags.go:64] FLAG: --storage-driver-password="root" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502199 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502202 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502205 2559 flags.go:64] FLAG: --storage-driver-user="root" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502207 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502210 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502213 2559 flags.go:64] FLAG: --system-cgroups="" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502217 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502222 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502225 2559 flags.go:64] FLAG: --tls-cert-file="" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502228 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 06 22:03:00.509025 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502232 2559 flags.go:64] FLAG: --tls-min-version="" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502234 2559 flags.go:64] FLAG: --tls-private-key-file="" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502237 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502240 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502243 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502245 2559 flags.go:64] FLAG: --v="2" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502249 2559 flags.go:64] FLAG: --version="false" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502253 2559 flags.go:64] FLAG: --vmodule="" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502256 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.502260 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502372 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502376 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502380 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502382 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502386 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502388 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502391 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502393 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502396 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502399 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502401 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502404 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:03:00.509664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502406 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502409 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502411 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502413 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502416 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502418 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502422 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502424 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502427 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502429 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502432 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502434 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502437 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502439 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502442 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502444 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502447 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502449 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502452 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502454 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:03:00.510250 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502456 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502460 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502463 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502465 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502468 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502482 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502485 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502488 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502490 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502494 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502497 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502500 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502503 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502506 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502509 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502512 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502514 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502517 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502521 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:03:00.510743 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502524 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502526 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502529 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502531 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502534 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502536 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502539 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502541 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502544 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502547 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502549 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502551 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502554 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502556 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502559 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502562 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502564 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502567 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502570 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:03:00.511293 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502573 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502576 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502579 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502581 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502584 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502586 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502588 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502591 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502593 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502595 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502598 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502601 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502606 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502608 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502611 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502614 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502617 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502620 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502622 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:03:00.511753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502625 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502627 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502630 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502632 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502635 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502637 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502639 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502642 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502645 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.502647 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.503183 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.509615 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.509631 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.510278 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.510286 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 06 22:03:00.512224 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510343 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510347 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510350 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510354 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510356 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510360 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510363 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510365 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510368 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510371 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510374 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510376 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510379 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510381 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510384 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510387 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510389 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510392 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510396 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:03:00.512603 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510400 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510402 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510406 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510409 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510412 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510414 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510417 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510420 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510422 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510425 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510429 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510431 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510434 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510437 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510439 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510442 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510444 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510447 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510449 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510452 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:03:00.513057 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510455 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510457 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510460 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510462 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510465 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510467 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510470 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510473 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510475 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510478 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510480 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510483 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510486 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510488 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510491 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510493 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510496 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510498 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510501 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510504 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:03:00.513578 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510507 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510509 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510512 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510515 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510518 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510521 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510523 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510526 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510528 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510531 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510533 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510536 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510538 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510541 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510543 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510546 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510548 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510551 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510554 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510556 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:03:00.514060 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510559 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510561 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510564 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510567 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510569 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510572 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510575 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510578 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510582 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510586 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510589 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510592 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510594 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510597 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510599 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510602 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510605 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510608 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510611 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:03:00.514617 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510614 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.510620 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510728 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510733 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510737 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510740 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510744 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510748 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510751 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510754 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510756 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510759 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510762 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510765 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510767 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:03:00.515249 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510770 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510772 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510775 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510778 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510780 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510783 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510786 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510788 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510791 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510793 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510796 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510799 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510801 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510804 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510806 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510810 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510813 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510816 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510818 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510821 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:03:00.515672 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510823 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510826 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510828 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510831 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510833 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510836 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510838 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510841 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510843 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510846 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510848 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510851 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510853 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510856 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510859 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510861 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510864 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510866 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510870 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:03:00.516301 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510873 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510875 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510877 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510880 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510882 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510885 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510887 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510890 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510892 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510896 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510898 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510901 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510903 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510906 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510908 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510910 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510913 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510915 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510918 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510920 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:03:00.516812 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510923 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510925 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510927 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510930 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510932 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510935 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510937 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510940 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510943 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510945 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510947 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510950 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510953 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510956 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510959 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510962 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510964 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510967 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510969 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:03:00.517637 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510972 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510974 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510976 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510980 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510982 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510985 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510987 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:00.510989 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.510994 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.511122 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.513072 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.514372 2559 server.go:1019] "Starting client certificate rotation" Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.514482 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 06 22:03:00.518201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.515137 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 06 22:03:00.518673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.516963 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 06 22:03:00.518673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.518106 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 06 22:03:00.526390 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.526375 2559 log.go:25] "Validated CRI v1 runtime API" Jun 06 22:03:00.530026 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.530010 2559 log.go:25] "Validated CRI v1 image API" Jun 06 22:03:00.534245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.534224 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 06 22:03:00.536471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.536454 2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b5d70fc9-85d1-4b83-ab22-ed1ee7d825a2:/dev/nvme0n1p3 dedcc645-a110-48a7-9f10-afb8690da0ed:/dev/nvme0n1p4] Jun 06 22:03:00.536550 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.536471 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 06 22:03:00.541376 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.541267 2559 manager.go:216] Machine: {Timestamp:2026-06-06 22:03:00.54008593 +0000 UTC m=+0.288650752 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099476 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b05589bafdd463f5524c184e1e7f1 SystemUUID:ec2b0558-9baf-dd46-3f55-24c184e1e7f1 BootID:7f15e2b9-bf5f-4239-98d4-009c1d1968ad Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:a4:8c:c2:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:a4:8c:c2:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:b9:b5:e7:7c:80 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 06 22:03:00.541376 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.541370 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 06 22:03:00.541493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.541444 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 06 22:03:00.543925 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.543908 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 06 22:03:00.544404 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.544380 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 06 22:03:00.544548 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.544408 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-77.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 06 22:03:00.545040 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.545029 2559 topology_manager.go:138] "Creating topology manager with none policy" Jun 06 22:03:00.545040 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.545041 2559 container_manager_linux.go:309] "Creating device plugin manager" Jun 06 22:03:00.545129 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.545049 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 06 22:03:00.545593 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.545582 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 06 22:03:00.546322 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.546312 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 06 22:03:00.548973 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.548953 2559 state_mem.go:36] "Initialized new in-memory state store" Jun 06 22:03:00.549072 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549063 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 06 22:03:00.549117 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549091 2559 kubelet.go:486] "Attempting to sync node with API server" Jun 06 22:03:00.549156 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549128 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 06 22:03:00.549156 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549143 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 06 22:03:00.549156 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549155 2559 kubelet.go:398] "Adding apiserver pod source" Jun 06 22:03:00.549232 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549163 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 06 22:03:00.549507 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549493 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 06 22:03:00.549549 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.549515 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 06 22:03:00.551588 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.551568 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 06 22:03:00.553243 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.553230 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 06 22:03:00.553286 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.553270 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 06 22:03:00.554245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554233 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 06 22:03:00.554282 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554258 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 06 22:03:00.554282 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554268 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 06 22:03:00.554282 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554278 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554292 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554301 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554314 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554321 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554331 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554337 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554345 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 06 22:03:00.554361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.554350 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 06 22:03:00.555050 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.555041 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 06 22:03:00.555050 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.555050 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 06 22:03:00.558588 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.558572 2559 server.go:1268] "Started kubelet" Jun 06 22:03:00.558656 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.558637 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 06 22:03:00.558822 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.558722 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 06 22:03:00.558913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.558900 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 06 22:03:00.559465 ip-10-0-136-77 systemd[1]: Started Kubernetes Kubelet. Jun 06 22:03:00.560166 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.559642 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-77.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 06 22:03:00.560380 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.560300 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 06 22:03:00.560430 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.560371 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 06 22:03:00.561150 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.561113 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 06 22:03:00.561736 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.561715 2559 server.go:310] "Adding debug handlers to kubelet server" Jun 06 22:03:00.563546 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.562762 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-77.ec2.internal.18b69d0cb41c75e1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-77.ec2.internal,UID:ip-10-0-136-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-77.ec2.internal,},FirstTimestamp:2026-06-06 22:03:00.558542305 +0000 UTC m=+0.307107017,LastTimestamp:2026-06-06 22:03:00.558542305 +0000 UTC m=+0.307107017,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-77.ec2.internal,}" Jun 06 22:03:00.565482 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.565465 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 06 22:03:00.565828 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.565806 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 06 22:03:00.566017 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.565752 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 06 22:03:00.566274 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.566250 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:00.567144 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567123 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 06 22:03:00.567144 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567143 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 06 22:03:00.567340 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567326 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 06 22:03:00.567420 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567395 2559 factory.go:55] Registering systemd factory Jun 06 22:03:00.567420 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567410 2559 factory.go:223] Registration of the systemd container factory successfully Jun 06 22:03:00.567515 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567397 2559 reconstruct.go:97] "Volume reconstruction finished" Jun 06 22:03:00.567515 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567465 2559 reconciler.go:29] "Reconciler: start to sync state" Jun 06 22:03:00.567721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567705 2559 factory.go:153] Registering CRI-O factory Jun 06 22:03:00.567771 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567724 2559 factory.go:223] Registration of the crio container factory successfully Jun 06 22:03:00.567814 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567783 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 06 22:03:00.567814 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567808 2559 factory.go:103] Registering Raw factory Jun 06 22:03:00.567899 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.567822 2559 manager.go:1196] Started watching for new ooms in manager Jun 06 22:03:00.568372 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.568355 2559 manager.go:349] Starting recovery of all containers Jun 06 22:03:00.570699 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.570471 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zcslc" Jun 06 22:03:00.576366 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.576191 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 06 22:03:00.577619 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.577594 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zcslc" Jun 06 22:03:00.580113 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.580065 2559 manager.go:354] Recovery completed Jun 06 22:03:00.581334 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.581259 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 06 22:03:00.581462 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.581443 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 06 22:03:00.581582 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.581567 2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 06 22:03:00.584499 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.584486 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.588296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588282 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.588357 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588309 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.588357 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588318 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.588843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588829 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 06 22:03:00.588843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588842 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 06 22:03:00.588934 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588860 2559 state_mem.go:36] "Initialized new in-memory state store" Jun 06 22:03:00.588934 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.588833 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.589615 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.589601 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.589673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.589627 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.589673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.589636 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.590064 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.590051 2559 policy_none.go:49] "None policy: Start" Jun 06 22:03:00.590146 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.590066 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 06 22:03:00.590146 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.590076 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 06 22:03:00.591203 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.591192 2559 policy_none.go:47] "Start" Jun 06 22:03:00.628769 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.628751 2559 manager.go:337] "Starting Device Plugin manager" Jun 06 22:03:00.628769 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.628779 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 06 22:03:00.628769 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.628789 2559 server.go:87] "Starting device plugin registration server" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.629028 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.629038 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.629128 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.629215 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.629225 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.629786 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.629844 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.644084 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.644142 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.644161 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.644191 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 06 22:03:00.655380 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.646659 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:03:00.730179 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.730140 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.731583 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.731567 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.731646 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.731598 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.731646 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.731611 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.731646 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.731638 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.740175 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.740161 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.740218 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.740186 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-77.ec2.internal\": node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:00.744286 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.744244 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal"] Jun 06 22:03:00.744331 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.744301 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.745968 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.745945 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.745968 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.745967 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.746116 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.745981 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.750317 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.750299 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.750425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.750403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.750484 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.750429 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.751043 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.751028 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.751139 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.751055 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.751139 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.751068 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.751139 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.751030 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.751287 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.751147 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.751287 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.751164 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.753428 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.753412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.753510 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.753444 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:03:00.754192 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.754171 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:03:00.754272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.754205 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:03:00.754272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.754214 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:03:00.755675 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.755658 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:00.768834 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.768814 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4c3b95f67e6ece35eb8b63e99f3c34ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"4c3b95f67e6ece35eb8b63e99f3c34ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.768903 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.768842 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c3b95f67e6ece35eb8b63e99f3c34ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"4c3b95f67e6ece35eb8b63e99f3c34ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.768903 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.768859 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6be6f86c99714a653cd42b51cc33e8d9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-77.ec2.internal\" (UID: \"6be6f86c99714a653cd42b51cc33e8d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.771728 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.771707 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-77.ec2.internal\" not found" node="ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.775696 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.775682 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-77.ec2.internal\" not found" node="ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.856066 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.856038 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:00.869687 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.869655 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6be6f86c99714a653cd42b51cc33e8d9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-77.ec2.internal\" (UID: \"6be6f86c99714a653cd42b51cc33e8d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.869762 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.869686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4c3b95f67e6ece35eb8b63e99f3c34ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"4c3b95f67e6ece35eb8b63e99f3c34ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.869762 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.869753 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6be6f86c99714a653cd42b51cc33e8d9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-77.ec2.internal\" (UID: \"6be6f86c99714a653cd42b51cc33e8d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.869825 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.869782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c3b95f67e6ece35eb8b63e99f3c34ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"4c3b95f67e6ece35eb8b63e99f3c34ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.869825 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.869730 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4c3b95f67e6ece35eb8b63e99f3c34ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"4c3b95f67e6ece35eb8b63e99f3c34ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.869825 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:00.869805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c3b95f67e6ece35eb8b63e99f3c34ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"4c3b95f67e6ece35eb8b63e99f3c34ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:00.956914 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:00.956877 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.057830 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.057737 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.078292 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.078268 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:01.082851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.082831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" Jun 06 22:03:01.158450 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.158418 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.258995 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.258974 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.359617 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.359539 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.460283 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.460245 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.477217 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.477191 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:03:01.514256 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.514225 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 06 22:03:01.514970 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.514406 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 06 22:03:01.514970 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.514430 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 06 22:03:01.560348 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.560314 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.580804 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.580768 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-05 21:58:00 +0000 UTC" deadline="2027-11-22 11:40:46.851262566 +0000 UTC" Jun 06 22:03:01.580804 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.580800 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12805h37m45.270466283s" Jun 06 22:03:01.633070 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:01.633036 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3b95f67e6ece35eb8b63e99f3c34ff.slice/crio-506b257e49fe86c690d22d658ff8222194be19c57658916557801154c50515a7 WatchSource:0}: Error finding container 506b257e49fe86c690d22d658ff8222194be19c57658916557801154c50515a7: Status 404 returned error can't find the container with id 506b257e49fe86c690d22d658ff8222194be19c57658916557801154c50515a7 Jun 06 22:03:01.633287 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:01.633264 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be6f86c99714a653cd42b51cc33e8d9.slice/crio-5841a7de278061f5fa86950c153cf4d8fe1b9ac876f0a2ce09c664341b5ff3a3 WatchSource:0}: Error finding container 5841a7de278061f5fa86950c153cf4d8fe1b9ac876f0a2ce09c664341b5ff3a3: Status 404 returned error can't find the container with id 5841a7de278061f5fa86950c153cf4d8fe1b9ac876f0a2ce09c664341b5ff3a3 Jun 06 22:03:01.637630 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.637614 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:03:01.647499 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.647456 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" event={"ID":"6be6f86c99714a653cd42b51cc33e8d9","Type":"ContainerStarted","Data":"5841a7de278061f5fa86950c153cf4d8fe1b9ac876f0a2ce09c664341b5ff3a3"} Jun 06 22:03:01.648433 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:01.648416 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" event={"ID":"4c3b95f67e6ece35eb8b63e99f3c34ff","Type":"ContainerStarted","Data":"506b257e49fe86c690d22d658ff8222194be19c57658916557801154c50515a7"} Jun 06 22:03:01.660596 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.660569 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.761112 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.761052 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.861665 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.861628 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:01.962218 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:01.962140 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:02.063090 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.063049 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" Jun 06 22:03:02.063343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.063319 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:03:02.075004 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.074975 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:03:02.167971 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.167790 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" Jun 06 22:03:02.178159 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.178133 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 06 22:03:02.179690 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.179473 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" Jun 06 22:03:02.185436 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.185410 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 06 22:03:02.550590 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.550561 2559 apiserver.go:52] "Watching apiserver" Jun 06 22:03:02.559336 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.559311 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 06 22:03:02.559710 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.559690 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-xh4kn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal","openshift-multus/network-metrics-daemon-978qx","openshift-network-diagnostics/network-check-target-z46ck","openshift-network-operator/iptables-alerter-nlbg9","kube-system/global-pull-secret-syncer-k6f4v","kube-system/konnectivity-agent-dqkkb","kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal","openshift-multus/multus-additional-cni-plugins-qjk7t","openshift-multus/multus-p6gg8","openshift-ovn-kubernetes/ovnkube-node-dxdx6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg","openshift-cluster-node-tuning-operator/tuned-z5mfs","openshift-dns/node-resolver-5v4lt"] Jun 06 22:03:02.562723 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.562704 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.564732 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.564709 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.566529 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.566512 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 06 22:03:02.566918 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.566899 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:02.567003 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.566975 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:02.569416 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.569291 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:02.569416 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.569364 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:02.569416 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.569395 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 06 22:03:02.569653 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.569540 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.569653 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.569567 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dp5xn\"" Jun 06 22:03:02.569653 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.569622 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xk8f8\"" Jun 06 22:03:02.570080 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.570052 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 06 22:03:02.573029 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.571442 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.573029 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.572406 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.574310 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.574291 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.574511 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.574482 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:02.574665 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.574650 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.576240 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.575919 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.576240 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.576129 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmsq8\"" Jun 06 22:03:02.576446 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.576425 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 06 22:03:02.577188 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.576782 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.578927 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.578904 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4b518bb-9e63-4eb4-b04b-4a045d66dbdf-agent-certs\") pod \"konnectivity-agent-dqkkb\" (UID: \"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf\") " pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.579036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.578939 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-modprobe-d\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.578963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysctl-conf\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.578995 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-var-lib-kubelet\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579017 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxqpl\" (UniqueName: \"kubernetes.io/projected/48b2b071-fc04-44b2-9506-8b95f5d12cfa-kube-api-access-bxqpl\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579047 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-run\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579111 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-host\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-tuned\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579163 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysconfig\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysctl-d\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-sys\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48b2b071-fc04-44b2-9506-8b95f5d12cfa-tmp\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k679\" (UniqueName: \"kubernetes.io/projected/7635dc64-f99d-4422-99ff-419efdcefc01-kube-api-access-7k679\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4b518bb-9e63-4eb4-b04b-4a045d66dbdf-konnectivity-ca\") pod \"konnectivity-agent-dqkkb\" (UID: \"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf\") " pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-kubernetes\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-systemd\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579326 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-lib-modules\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579346 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:02.579626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.579361 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.581254 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.581237 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.583601 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.583584 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.585943 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.585922 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.588278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.588258 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.594346 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.594329 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s9wd7\"" Jun 06 22:03:02.595413 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.595397 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.595763 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.595746 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.596357 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.596340 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 06 22:03:02.596841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.596826 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-txkvz\"" Jun 06 22:03:02.597209 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.597192 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.597384 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.597365 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 06 22:03:02.597713 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.597697 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.598014 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.597992 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.598273 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.598257 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 06 22:03:02.598743 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.598701 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 06 22:03:02.599014 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.598994 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xcg98\"" Jun 06 22:03:02.599608 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.599595 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 06 22:03:02.601022 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.601006 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.601120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.601059 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.610024 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610008 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xb4wr\"" Jun 06 22:03:02.610246 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610227 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.610342 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610033 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 06 22:03:02.610403 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610345 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 06 22:03:02.610403 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610077 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 06 22:03:02.610499 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610114 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fqxbh\"" Jun 06 22:03:02.610657 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610639 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 06 22:03:02.610782 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610760 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 06 22:03:02.610867 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610823 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 06 22:03:02.611008 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.610992 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5nr5x\"" Jun 06 22:03:02.611074 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.611014 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 06 22:03:02.611198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.611181 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 06 22:03:02.642058 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.642035 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rxfst" Jun 06 22:03:02.650957 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.650922 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rxfst" Jun 06 22:03:02.668477 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.668451 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 06 22:03:02.680175 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4b518bb-9e63-4eb4-b04b-4a045d66dbdf-agent-certs\") pod \"konnectivity-agent-dqkkb\" (UID: \"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf\") " pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.680297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.680297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-cni-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.680297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680235 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-run-ovn-kubernetes\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.680297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680256 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-socket-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680302 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-sys-fs\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680350 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-tuned\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680386 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-var-lib-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680413 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysconfig\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680506 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysconfig\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680439 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k679\" (UniqueName: \"kubernetes.io/projected/7635dc64-f99d-4422-99ff-419efdcefc01-kube-api-access-7k679\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680563 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17ca1add-4025-4304-b1ef-97df8c163066-iptables-alerter-script\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680556 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 06 22:03:02.680612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-cni-netd\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.681038 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680720 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-env-overrides\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.681038 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680748 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:02.681038 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.680908 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:02.681038 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.680988 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:03.18096527 +0000 UTC m=+2.929529968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:02.681271 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.680773 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-cnibin\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.681271 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681227 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-run-netns\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.681271 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681253 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96tbb\" (UniqueName: \"kubernetes.io/projected/460c7697-a783-40cb-a171-2abe6550da31-kube-api-access-96tbb\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681307 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-modprobe-d\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681378 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681409 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-system-cni-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681435 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-k8s-cni-cncf-io\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681435 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-modprobe-d\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.681471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681461 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-cni-multus\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-run\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17ca1add-4025-4304-b1ef-97df8c163066-host-slash\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681641 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-run\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681682 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-dbus\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681750 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c365b91d-ec96-452a-a635-fffdaf8f6f2c-serviceca\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-os-release\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.681790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681794 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-slash\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681808 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovnkube-config\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysctl-d\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681840 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48b2b071-fc04-44b2-9506-8b95f5d12cfa-tmp\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-log-socket\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681881 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-registration-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681895 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-system-cni-dir\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-tmp-dir\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681933 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnb8z\" (UniqueName: \"kubernetes.io/projected/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-kube-api-access-jnb8z\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681952 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-cnibin\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681952 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysctl-d\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-kubelet\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.681982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovn-node-metrics-cert\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682007 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxsx4\" (UniqueName: \"kubernetes.io/projected/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-kube-api-access-rxsx4\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682030 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-device-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682047 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxqpl\" (UniqueName: \"kubernetes.io/projected/48b2b071-fc04-44b2-9506-8b95f5d12cfa-kube-api-access-bxqpl\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682078 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-hosts-file\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.682193 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682129 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-netns\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682153 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-etc-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls5pt\" (UniqueName: \"kubernetes.io/projected/c365b91d-ec96-452a-a635-fffdaf8f6f2c-kube-api-access-ls5pt\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-socket-dir-parent\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682286 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-kubelet\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-ovn\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682336 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682385 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-sys\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682418 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-hostroot\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682440 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-cni-bin\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682491 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-sys\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4b518bb-9e63-4eb4-b04b-4a045d66dbdf-konnectivity-ca\") pod \"konnectivity-agent-dqkkb\" (UID: \"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf\") " pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682546 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-systemd\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682561 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-lib-modules\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682615 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-os-release\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.682914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682661 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c365b91d-ec96-452a-a635-fffdaf8f6f2c-host\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682659 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-systemd\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e51cf442-9dc6-436e-93f3-aaf840764973-cni-binary-copy\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682726 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-multus-certs\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovnkube-script-lib\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysctl-conf\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-var-lib-kubelet\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682869 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-kubelet-config\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682930 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-var-lib-kubelet\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.682974 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-cni-bin\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683003 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4b518bb-9e63-4eb4-b04b-4a045d66dbdf-konnectivity-ca\") pod \"konnectivity-agent-dqkkb\" (UID: \"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf\") " pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683049 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-lib-modules\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-host\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683088 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683112 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-host\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683125 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-sysctl-conf\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683135 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-conf-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.683644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-etc-kubernetes\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e51cf442-9dc6-436e-93f3-aaf840764973-multus-daemon-config\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-systemd\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-kubernetes\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683283 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjmvn\" (UniqueName: \"kubernetes.io/projected/17ca1add-4025-4304-b1ef-97df8c163066-kube-api-access-mjmvn\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683296 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-kubernetes\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5x45\" (UniqueName: \"kubernetes.io/projected/b24c7544-f928-4f19-ba03-5f51f474f7f8-kube-api-access-t5x45\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjdtt\" (UniqueName: \"kubernetes.io/projected/e51cf442-9dc6-436e-93f3-aaf840764973-kube-api-access-kjdtt\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683361 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-systemd-units\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-node-log\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.684120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.683416 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.684521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.684299 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48b2b071-fc04-44b2-9506-8b95f5d12cfa-tmp\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.684521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.684348 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48b2b071-fc04-44b2-9506-8b95f5d12cfa-etc-tuned\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.684583 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.684555 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4b518bb-9e63-4eb4-b04b-4a045d66dbdf-agent-certs\") pod \"konnectivity-agent-dqkkb\" (UID: \"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf\") " pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.691478 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.691457 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:02.691478 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.691481 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:02.691657 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.691492 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:02.691657 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.691546 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:03.191528781 +0000 UTC m=+2.940093478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:02.693011 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.692992 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k679\" (UniqueName: \"kubernetes.io/projected/7635dc64-f99d-4422-99ff-419efdcefc01-kube-api-access-7k679\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:02.693791 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.693769 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxqpl\" (UniqueName: \"kubernetes.io/projected/48b2b071-fc04-44b2-9506-8b95f5d12cfa-kube-api-access-bxqpl\") pod \"tuned-z5mfs\" (UID: \"48b2b071-fc04-44b2-9506-8b95f5d12cfa\") " pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.783879 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.783842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjmvn\" (UniqueName: \"kubernetes.io/projected/17ca1add-4025-4304-b1ef-97df8c163066-kube-api-access-mjmvn\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.783899 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5x45\" (UniqueName: \"kubernetes.io/projected/b24c7544-f928-4f19-ba03-5f51f474f7f8-kube-api-access-t5x45\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.783931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjdtt\" (UniqueName: \"kubernetes.io/projected/e51cf442-9dc6-436e-93f3-aaf840764973-kube-api-access-kjdtt\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.783958 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-systemd-units\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.783983 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-node-log\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784035 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.784049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784040 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-systemd-units\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-cni-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-run-ovn-kubernetes\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-socket-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784156 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-cni-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784168 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-sys-fs\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784156 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-node-log\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784194 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-run-ovn-kubernetes\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.784195 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784197 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-var-lib-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784243 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784252 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-sys-fs\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:02.784265 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:03.284244421 +0000 UTC m=+3.032809133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784274 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-var-lib-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784289 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-socket-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17ca1add-4025-4304-b1ef-97df8c163066-iptables-alerter-script\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-cni-netd\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.784478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784359 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-env-overrides\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784416 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-cni-netd\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784448 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-cnibin\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-run-netns\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96tbb\" (UniqueName: \"kubernetes.io/projected/460c7697-a783-40cb-a171-2abe6550da31-kube-api-access-96tbb\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784549 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-cnibin\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784556 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-run-netns\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784579 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784604 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-system-cni-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-k8s-cni-cncf-io\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784676 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-k8s-cni-cncf-io\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784682 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-system-cni-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784706 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-cni-multus\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784732 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17ca1add-4025-4304-b1ef-97df8c163066-host-slash\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784744 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-cni-multus\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.785272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-dbus\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784799 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17ca1add-4025-4304-b1ef-97df8c163066-host-slash\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784812 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784825 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-env-overrides\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784840 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c365b91d-ec96-452a-a635-fffdaf8f6f2c-serviceca\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784863 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-dbus\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784873 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17ca1add-4025-4304-b1ef-97df8c163066-iptables-alerter-script\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784867 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-os-release\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784909 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-slash\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovnkube-config\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-log-socket\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784974 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-os-release\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.784995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-registration-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785004 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-slash\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785021 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-system-cni-dir\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785049 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-tmp-dir\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-log-socket\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnb8z\" (UniqueName: \"kubernetes.io/projected/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-kube-api-access-jnb8z\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785079 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-cni-binary-copy\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785117 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-cnibin\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-kubelet\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovn-node-metrics-cert\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785185 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-cnibin\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785199 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxsx4\" (UniqueName: \"kubernetes.io/projected/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-kube-api-access-rxsx4\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785224 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-device-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785251 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-hosts-file\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-netns\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785307 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-etc-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls5pt\" (UniqueName: \"kubernetes.io/projected/c365b91d-ec96-452a-a635-fffdaf8f6f2c-kube-api-access-ls5pt\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785385 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-socket-dir-parent\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-kubelet\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785433 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-tmp-dir\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785472 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-ovn\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.786833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-ovn\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785490 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-device-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785499 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785536 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-hostroot\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785577 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-netns\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785589 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-etc-openvswitch\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785119 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-system-cni-dir\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785584 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-cni-bin\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785148 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-registration-dir\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785626 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-cni-bin\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785633 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-hostroot\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785681 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-hosts-file\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785690 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-host-kubelet\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785581 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-kubelet\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785717 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-socket-dir-parent\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.787506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785825 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-os-release\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785913 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/460c7697-a783-40cb-a171-2abe6550da31-etc-selinux\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c365b91d-ec96-452a-a635-fffdaf8f6f2c-host\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785949 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e51cf442-9dc6-436e-93f3-aaf840764973-cni-binary-copy\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785970 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-os-release\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.785972 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-multus-certs\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786004 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-run-multus-certs\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovnkube-script-lib\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786043 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-kubelet-config\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786059 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c365b91d-ec96-452a-a635-fffdaf8f6f2c-serviceca\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-cni-bin\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786136 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-kubelet-config\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786044 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c365b91d-ec96-452a-a635-fffdaf8f6f2c-host\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786145 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b24c7544-f928-4f19-ba03-5f51f474f7f8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786211 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-host-var-lib-cni-bin\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.788293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-conf-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-etc-kubernetes\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786287 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b24c7544-f928-4f19-ba03-5f51f474f7f8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e51cf442-9dc6-436e-93f3-aaf840764973-multus-daemon-config\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-systemd\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786331 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-etc-kubernetes\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786332 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e51cf442-9dc6-436e-93f3-aaf840764973-multus-conf-dir\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786396 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-run-systemd\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786431 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovnkube-config\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786549 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e51cf442-9dc6-436e-93f3-aaf840764973-cni-binary-copy\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.786672 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e51cf442-9dc6-436e-93f3-aaf840764973-multus-daemon-config\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.787114 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovnkube-script-lib\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.788748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.788559 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-ovn-node-metrics-cert\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.795905 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.795871 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls5pt\" (UniqueName: \"kubernetes.io/projected/c365b91d-ec96-452a-a635-fffdaf8f6f2c-kube-api-access-ls5pt\") pod \"node-ca-xh4kn\" (UID: \"c365b91d-ec96-452a-a635-fffdaf8f6f2c\") " pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.796363 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.796341 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjdtt\" (UniqueName: \"kubernetes.io/projected/e51cf442-9dc6-436e-93f3-aaf840764973-kube-api-access-kjdtt\") pod \"multus-p6gg8\" (UID: \"e51cf442-9dc6-436e-93f3-aaf840764973\") " pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.796463 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.796367 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96tbb\" (UniqueName: \"kubernetes.io/projected/460c7697-a783-40cb-a171-2abe6550da31-kube-api-access-96tbb\") pod \"aws-ebs-csi-driver-node-cgmlg\" (UID: \"460c7697-a783-40cb-a171-2abe6550da31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:02.796463 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.796382 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5x45\" (UniqueName: \"kubernetes.io/projected/b24c7544-f928-4f19-ba03-5f51f474f7f8-kube-api-access-t5x45\") pod \"multus-additional-cni-plugins-qjk7t\" (UID: \"b24c7544-f928-4f19-ba03-5f51f474f7f8\") " pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.796463 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.796384 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxsx4\" (UniqueName: \"kubernetes.io/projected/e2fd752e-f7f8-44e5-beaa-5aed253e7e19-kube-api-access-rxsx4\") pod \"ovnkube-node-dxdx6\" (UID: \"e2fd752e-f7f8-44e5-beaa-5aed253e7e19\") " pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.796657 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.796640 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnb8z\" (UniqueName: \"kubernetes.io/projected/bb57a9c4-fe3e-4f7f-b452-011eee4d5173-kube-api-access-jnb8z\") pod \"node-resolver-5v4lt\" (UID: \"bb57a9c4-fe3e-4f7f-b452-011eee4d5173\") " pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.797458 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.797431 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjmvn\" (UniqueName: \"kubernetes.io/projected/17ca1add-4025-4304-b1ef-97df8c163066-kube-api-access-mjmvn\") pod \"iptables-alerter-nlbg9\" (UID: \"17ca1add-4025-4304-b1ef-97df8c163066\") " pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.814283 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.814222 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:03:02.874504 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.874466 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:02.881562 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.881537 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" Jun 06 22:03:02.892370 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.892345 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlbg9" Jun 06 22:03:02.895826 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.895810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" Jun 06 22:03:02.904353 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.904335 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xh4kn" Jun 06 22:03:02.910001 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.909981 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5v4lt" Jun 06 22:03:02.916623 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.916606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p6gg8" Jun 06 22:03:02.924249 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.924231 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:02.928920 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:02.928899 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" Jun 06 22:03:03.189354 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.189268 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:03.189490 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.189384 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:03.189490 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.189443 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:04.189428372 +0000 UTC m=+3.937993070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:03.290278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.290243 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:03.290425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.290306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:03.290425 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.290388 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:03.290528 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.290431 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:03.290528 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.290447 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:03.290528 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.290455 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:04.290438675 +0000 UTC m=+4.039003372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:03.290528 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.290460 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:03.290528 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:03.290508 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:04.290491108 +0000 UTC m=+4.039055811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:03.489409 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.489383 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b2b071_fc04_44b2_9506_8b95f5d12cfa.slice/crio-053b0433f2cfcb863ad92230a256c2aa20a9eff8f30bb91ddee620b274b142ce WatchSource:0}: Error finding container 053b0433f2cfcb863ad92230a256c2aa20a9eff8f30bb91ddee620b274b142ce: Status 404 returned error can't find the container with id 053b0433f2cfcb863ad92230a256c2aa20a9eff8f30bb91ddee620b274b142ce Jun 06 22:03:03.490825 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.490790 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b518bb_9e63_4eb4_b04b_4a045d66dbdf.slice/crio-20fe195c4700f6c60fb0254a3f4a57e1d508a39a77bc18260232ab6f582343ea WatchSource:0}: Error finding container 20fe195c4700f6c60fb0254a3f4a57e1d508a39a77bc18260232ab6f582343ea: Status 404 returned error can't find the container with id 20fe195c4700f6c60fb0254a3f4a57e1d508a39a77bc18260232ab6f582343ea Jun 06 22:03:03.492011 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.491971 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ca1add_4025_4304_b1ef_97df8c163066.slice/crio-5243ec2168c24126efefad1fa03ec6f0776f9a4f14e8781a060170ca1727ce90 WatchSource:0}: Error finding container 5243ec2168c24126efefad1fa03ec6f0776f9a4f14e8781a060170ca1727ce90: Status 404 returned error can't find the container with id 5243ec2168c24126efefad1fa03ec6f0776f9a4f14e8781a060170ca1727ce90 Jun 06 22:03:03.514204 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.514173 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24c7544_f928_4f19_ba03_5f51f474f7f8.slice/crio-0b7cb1a7c21b5d3de4a84510f46a9da4f3b5d5a129efde3eaf20df1b9b6693a2 WatchSource:0}: Error finding container 0b7cb1a7c21b5d3de4a84510f46a9da4f3b5d5a129efde3eaf20df1b9b6693a2: Status 404 returned error can't find the container with id 0b7cb1a7c21b5d3de4a84510f46a9da4f3b5d5a129efde3eaf20df1b9b6693a2 Jun 06 22:03:03.518815 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.518786 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb57a9c4_fe3e_4f7f_b452_011eee4d5173.slice/crio-3f1ec9d0515d8ec88ea59e16e9b6c98f03cf7f80e553da15ee79c87e5b1d8c76 WatchSource:0}: Error finding container 3f1ec9d0515d8ec88ea59e16e9b6c98f03cf7f80e553da15ee79c87e5b1d8c76: Status 404 returned error can't find the container with id 3f1ec9d0515d8ec88ea59e16e9b6c98f03cf7f80e553da15ee79c87e5b1d8c76 Jun 06 22:03:03.519003 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.518984 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc365b91d_ec96_452a_a635_fffdaf8f6f2c.slice/crio-16359b5e4fa50608d7a52623fe797adb4a13cb50d1d4235287408effdcdcb6eb WatchSource:0}: Error finding container 16359b5e4fa50608d7a52623fe797adb4a13cb50d1d4235287408effdcdcb6eb: Status 404 returned error can't find the container with id 16359b5e4fa50608d7a52623fe797adb4a13cb50d1d4235287408effdcdcb6eb Jun 06 22:03:03.519829 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.519808 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51cf442_9dc6_436e_93f3_aaf840764973.slice/crio-4523d2fb122c5f6cceb88642067230fba7d78ed2560982088e2a2e016520782c WatchSource:0}: Error finding container 4523d2fb122c5f6cceb88642067230fba7d78ed2560982088e2a2e016520782c: Status 404 returned error can't find the container with id 4523d2fb122c5f6cceb88642067230fba7d78ed2560982088e2a2e016520782c Jun 06 22:03:03.520538 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.520516 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460c7697_a783_40cb_a171_2abe6550da31.slice/crio-d74a10e9cdcabcadf517e29444938474973dc70a5163da8e3a6a486891b2a72a WatchSource:0}: Error finding container d74a10e9cdcabcadf517e29444938474973dc70a5163da8e3a6a486891b2a72a: Status 404 returned error can't find the container with id d74a10e9cdcabcadf517e29444938474973dc70a5163da8e3a6a486891b2a72a Jun 06 22:03:03.522706 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:03.521381 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fd752e_f7f8_44e5_beaa_5aed253e7e19.slice/crio-4e310b358bc733ceef05f07474bf0ef682d3471cd287f1d6e3114294254f64ee WatchSource:0}: Error finding container 4e310b358bc733ceef05f07474bf0ef682d3471cd287f1d6e3114294254f64ee: Status 404 returned error can't find the container with id 4e310b358bc733ceef05f07474bf0ef682d3471cd287f1d6e3114294254f64ee Jun 06 22:03:03.654929 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.654769 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-05 21:58:02 +0000 UTC" deadline="2028-03-11 00:22:02.180719036 +0000 UTC" Jun 06 22:03:03.654929 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.654923 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15434h18m58.525798729s" Jun 06 22:03:03.657249 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.657222 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" event={"ID":"6be6f86c99714a653cd42b51cc33e8d9","Type":"ContainerStarted","Data":"b2068d42c709f96bec2ef669f1674bae007453de3ff32709b0ebd667e42633e0"} Jun 06 22:03:03.659241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.659215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"4e310b358bc733ceef05f07474bf0ef682d3471cd287f1d6e3114294254f64ee"} Jun 06 22:03:03.660129 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.660109 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5v4lt" event={"ID":"bb57a9c4-fe3e-4f7f-b452-011eee4d5173","Type":"ContainerStarted","Data":"3f1ec9d0515d8ec88ea59e16e9b6c98f03cf7f80e553da15ee79c87e5b1d8c76"} Jun 06 22:03:03.661634 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.661613 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlbg9" event={"ID":"17ca1add-4025-4304-b1ef-97df8c163066","Type":"ContainerStarted","Data":"5243ec2168c24126efefad1fa03ec6f0776f9a4f14e8781a060170ca1727ce90"} Jun 06 22:03:03.663990 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.663965 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" event={"ID":"48b2b071-fc04-44b2-9506-8b95f5d12cfa","Type":"ContainerStarted","Data":"053b0433f2cfcb863ad92230a256c2aa20a9eff8f30bb91ddee620b274b142ce"} Jun 06 22:03:03.664964 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.664944 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" event={"ID":"460c7697-a783-40cb-a171-2abe6550da31","Type":"ContainerStarted","Data":"d74a10e9cdcabcadf517e29444938474973dc70a5163da8e3a6a486891b2a72a"} Jun 06 22:03:03.665869 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.665848 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p6gg8" event={"ID":"e51cf442-9dc6-436e-93f3-aaf840764973","Type":"ContainerStarted","Data":"4523d2fb122c5f6cceb88642067230fba7d78ed2560982088e2a2e016520782c"} Jun 06 22:03:03.668680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.668660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xh4kn" event={"ID":"c365b91d-ec96-452a-a635-fffdaf8f6f2c","Type":"ContainerStarted","Data":"16359b5e4fa50608d7a52623fe797adb4a13cb50d1d4235287408effdcdcb6eb"} Jun 06 22:03:03.671528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.671482 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" podStartSLOduration=1.6714702030000002 podStartE2EDuration="1.671470203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:03.670887968 +0000 UTC m=+3.419452690" watchObservedRunningTime="2026-06-06 22:03:03.671470203 +0000 UTC m=+3.420034921" Jun 06 22:03:03.672413 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.672393 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerStarted","Data":"0b7cb1a7c21b5d3de4a84510f46a9da4f3b5d5a129efde3eaf20df1b9b6693a2"} Jun 06 22:03:03.675955 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:03.675938 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dqkkb" event={"ID":"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf","Type":"ContainerStarted","Data":"20fe195c4700f6c60fb0254a3f4a57e1d508a39a77bc18260232ab6f582343ea"} Jun 06 22:03:04.198020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.197373 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:04.198020 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.197539 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:04.198020 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.197605 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:06.197584928 +0000 UTC m=+5.946149644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:04.298245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.298198 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:04.298430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.298283 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:04.298498 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.298447 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:04.298498 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.298469 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:04.298498 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.298482 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:04.298639 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.298540 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:06.298522131 +0000 UTC m=+6.047086841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:04.298956 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.298925 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:04.299013 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.298979 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:06.298963532 +0000 UTC m=+6.047528284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:04.542783 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.542706 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:03:04.647693 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.647668 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:04.647831 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.647797 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:04.648220 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.648198 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:04.648345 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.648292 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:04.648457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.648357 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:04.648457 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:04.648420 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:04.660184 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.655369 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-05 21:58:02 +0000 UTC" deadline="2028-01-08 22:35:20.075303005 +0000 UTC" Jun 06 22:03:04.660184 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.655419 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13944h32m15.419888519s" Jun 06 22:03:04.710049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.710013 2559 generic.go:356] "Generic (PLEG): container finished" podID="4c3b95f67e6ece35eb8b63e99f3c34ff" containerID="aca066b2c9efd7fc657d7c30fa9d63910414096eed1aaa4633f4558de02a017e" exitCode=0 Jun 06 22:03:04.711011 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:04.710984 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" event={"ID":"4c3b95f67e6ece35eb8b63e99f3c34ff","Type":"ContainerDied","Data":"aca066b2c9efd7fc657d7c30fa9d63910414096eed1aaa4633f4558de02a017e"} Jun 06 22:03:05.728972 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:05.728290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" event={"ID":"4c3b95f67e6ece35eb8b63e99f3c34ff","Type":"ContainerStarted","Data":"c294cbb2d91d30ac941e6ef3128b373e83af201db8a13040bfb727891fb71865"} Jun 06 22:03:06.216145 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:06.215566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:06.216145 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.215722 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:06.216145 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.215782 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:10.21576825 +0000 UTC m=+9.964332947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:06.316576 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:06.316532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:06.316756 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:06.316607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:06.316827 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.316780 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:06.316827 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.316802 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:06.316827 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.316815 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:06.316964 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.316875 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:10.316855654 +0000 UTC m=+10.065420357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:06.317317 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.317268 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:06.317397 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.317327 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:10.317312181 +0000 UTC m=+10.065876880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:06.646449 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:06.646379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:06.646598 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.646504 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:06.646863 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:06.646843 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:06.646978 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.646948 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:06.647136 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:06.647001 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:06.647136 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:06.647070 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:08.645002 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:08.644805 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:08.645002 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:08.644810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:08.645002 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:08.644946 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:08.645002 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:08.644960 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:08.645641 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:08.645087 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:08.645641 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:08.645175 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:10.253111 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:10.253063 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:10.253517 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.253215 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:10.253517 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.253269 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:18.253255748 +0000 UTC m=+18.001820445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:10.353857 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:10.353822 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:10.354030 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:10.353885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:10.354030 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.354002 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:10.354030 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.354022 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:10.354245 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.354036 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:10.354245 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.354050 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:10.354245 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.354064 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:18.354045932 +0000 UTC m=+18.102610631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:10.354245 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.354083 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:18.354072444 +0000 UTC m=+18.102637162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:10.647834 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:10.647776 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:10.648108 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.648065 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:10.648552 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:10.648509 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:10.648652 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.648600 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:10.648652 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:10.648646 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:10.648754 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:10.648708 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:12.646235 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:12.645244 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:12.646235 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:12.645361 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:12.646235 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:12.645744 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:12.646235 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:12.645819 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:12.646809 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:12.646325 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:12.646809 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:12.646414 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:14.645282 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:14.645253 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:14.645282 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:14.645267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:14.645776 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:14.645256 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:14.645776 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:14.645382 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:14.645776 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:14.645461 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:14.645776 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:14.645571 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:16.644672 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:16.644634 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:16.645144 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:16.644646 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:16.645144 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:16.644778 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:16.645144 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:16.644652 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:16.645144 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:16.644856 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:16.645144 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:16.644904 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:18.313166 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:18.313130 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:18.313588 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.313240 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:18.313588 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.313321 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:34.313299298 +0000 UTC m=+34.061863998 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:18.414142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:18.414092 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:18.414307 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:18.414166 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:18.414307 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.414247 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:18.414397 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.414317 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:34.414295656 +0000 UTC m=+34.162860354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:18.414397 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.414322 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:18.414397 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.414343 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:18.414397 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.414358 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:18.414540 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.414407 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:34.4143914 +0000 UTC m=+34.162956102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:18.646183 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:18.644633 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:18.646183 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.644777 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:18.646183 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:18.644634 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:18.646183 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.644855 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:18.646183 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:18.644642 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:18.646183 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:18.644927 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:20.645387 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:20.645354 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:20.645927 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:20.645442 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:20.645927 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:20.645463 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:20.645927 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:20.645518 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:20.645927 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:20.645598 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:20.645927 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:20.645694 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:22.645490 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.645313 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:22.646297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.645374 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:22.646297 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:22.645562 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:22.646297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.645414 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:22.646297 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:22.645623 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:22.646297 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:22.645710 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:22.762679 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.762427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5v4lt" event={"ID":"bb57a9c4-fe3e-4f7f-b452-011eee4d5173","Type":"ContainerStarted","Data":"30839842bf3e8cbc058aec03955e15599d81d7a4c73991ebdf23a779219b7050"} Jun 06 22:03:22.766557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.766220 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" event={"ID":"48b2b071-fc04-44b2-9506-8b95f5d12cfa","Type":"ContainerStarted","Data":"4115f4e2bb2d830ba90673628a007b746b5730ca1aaaf9c50f627ad6bbf49e05"} Jun 06 22:03:22.767816 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.767788 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" event={"ID":"460c7697-a783-40cb-a171-2abe6550da31","Type":"ContainerStarted","Data":"5c7002889c0eca6e1c86c29fa768aeae5f0010f6d3e1abc7f972eae6320af994"} Jun 06 22:03:22.769925 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.769897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p6gg8" event={"ID":"e51cf442-9dc6-436e-93f3-aaf840764973","Type":"ContainerStarted","Data":"53578e2a4673ad67d3cdc34021d7db2797e10e08fa4b3cebcc611bc31199d9a4"} Jun 06 22:03:22.771584 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.771559 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xh4kn" event={"ID":"c365b91d-ec96-452a-a635-fffdaf8f6f2c","Type":"ContainerStarted","Data":"e2b7ef9e56ab3368854dfb6e95321ee7599ab9587ed18bbfd31dbd1c170a9960"} Jun 06 22:03:22.773537 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.773512 2559 generic.go:356] "Generic (PLEG): container finished" podID="b24c7544-f928-4f19-ba03-5f51f474f7f8" containerID="b0944526011d8583d9063694b0d68f8bcad8c5b34dd3a73f1a16b35bbb555bb3" exitCode=0 Jun 06 22:03:22.773660 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.773598 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerDied","Data":"b0944526011d8583d9063694b0d68f8bcad8c5b34dd3a73f1a16b35bbb555bb3"} Jun 06 22:03:22.775236 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.775157 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dqkkb" event={"ID":"c4b518bb-9e63-4eb4-b04b-4a045d66dbdf","Type":"ContainerStarted","Data":"93b24627609e1a1384d8421f79d76d8d8b38f90c0bdc0ea446b360cafb0524b1"} Jun 06 22:03:22.778191 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.778173 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:03:22.778609 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.778586 2559 generic.go:356] "Generic (PLEG): container finished" podID="e2fd752e-f7f8-44e5-beaa-5aed253e7e19" containerID="0d4823a8c26ac1aa680b508cd4f1cdaec5db2498b0eebfa1057fb6e53da5cd0b" exitCode=1 Jun 06 22:03:22.778691 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.778618 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"5a0b404e08764867e8fbaf4d7364badf790036be98347c98a067f0c7927563d0"} Jun 06 22:03:22.778763 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.778717 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"a9300f6e4938ed4acd63093086bcf91425cdd3e3b9e39274340acfef4caa88f2"} Jun 06 22:03:22.778825 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.778777 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerDied","Data":"0d4823a8c26ac1aa680b508cd4f1cdaec5db2498b0eebfa1057fb6e53da5cd0b"} Jun 06 22:03:22.778874 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.778830 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"79a9f83478ae6494dcdea38704f8abad692a64f33b1a128d17407b9f2cba0020"} Jun 06 22:03:22.779740 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.779696 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5v4lt" podStartSLOduration=3.940624809 podStartE2EDuration="22.779679695s" totalImagesPullingTime="18.839054886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:22.77906791 +0000 UTC m=+22.527632628" watchObservedRunningTime="2026-06-06 22:03:22.779679695 +0000 UTC m=+22.528244417" Jun 06 22:03:22.779861 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.779836 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" podStartSLOduration=20.779830417 podStartE2EDuration="20.779830417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:05.744639866 +0000 UTC m=+5.493204603" watchObservedRunningTime="2026-06-06 22:03:22.779830417 +0000 UTC m=+22.528395138" Jun 06 22:03:22.800226 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.800179 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p6gg8" podStartSLOduration=4.168594102 podStartE2EDuration="22.800159739s" totalImagesPullingTime="18.631565637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:22.799501351 +0000 UTC m=+22.548066083" watchObservedRunningTime="2026-06-06 22:03:22.800159739 +0000 UTC m=+22.548724458" Jun 06 22:03:22.814042 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.814004 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dqkkb" podStartSLOduration=4.229004864 podStartE2EDuration="22.813985105s" totalImagesPullingTime="18.584980241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:22.813733703 +0000 UTC m=+22.562298423" watchObservedRunningTime="2026-06-06 22:03:22.813985105 +0000 UTC m=+22.562549826" Jun 06 22:03:22.832785 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.832733 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z5mfs" podStartSLOduration=4.247736969 podStartE2EDuration="22.832714119s" totalImagesPullingTime="18.58497715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:22.832146152 +0000 UTC m=+22.580710888" watchObservedRunningTime="2026-06-06 22:03:22.832714119 +0000 UTC m=+22.581278839" Jun 06 22:03:22.870152 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:22.870111 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xh4kn" podStartSLOduration=4.280340443 podStartE2EDuration="22.870078295s" totalImagesPullingTime="18.589737852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:22.869459522 +0000 UTC m=+22.618024241" watchObservedRunningTime="2026-06-06 22:03:22.870078295 +0000 UTC m=+22.618643013" Jun 06 22:03:23.389851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.389807 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:23.390421 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.390404 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:23.729435 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.729367 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 06 22:03:23.784329 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.784295 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:03:23.784814 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.784782 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"1c8728ac9a1b656c1bfa33c32959496f8265a81b1b761f31684a8f9ea47395b3"} Jun 06 22:03:23.784924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.784868 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"b0cfa687638003e70ce8f0bb0a18f22c7e60bd6556ecd45f0c37d2cc5bfc82d7"} Jun 06 22:03:23.786405 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.786373 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlbg9" event={"ID":"17ca1add-4025-4304-b1ef-97df8c163066","Type":"ContainerStarted","Data":"c5cc70298e818fb5cbadc6eb8fb9a1415b8abf56932bddb306df4e977de45057"} Jun 06 22:03:23.788448 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.788411 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" event={"ID":"460c7697-a783-40cb-a171-2abe6550da31","Type":"ContainerStarted","Data":"7063f8e6d411cceb4a524afcdd81474668684568224c9b7bc008d08d92e55fb7"} Jun 06 22:03:23.789185 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.789082 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:23.789554 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.789534 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dqkkb" Jun 06 22:03:23.800427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:23.800388 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nlbg9" podStartSLOduration=5.201177178 podStartE2EDuration="23.800372607s" totalImagesPullingTime="18.599195429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:23.799942398 +0000 UTC m=+23.548507116" watchObservedRunningTime="2026-06-06 22:03:23.800372607 +0000 UTC m=+23.548937329" Jun 06 22:03:24.640706 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:24.640604 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-06T22:03:23.729382772Z","UUID":"44880cf7-37a9-4952-a000-7b9e14fe3dbf","Handler":null,"Name":"","Endpoint":""} Jun 06 22:03:24.642503 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:24.642471 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 06 22:03:24.642618 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:24.642511 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 06 22:03:24.644947 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:24.644908 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:24.645060 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:24.644915 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:24.645060 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:24.645043 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:24.645202 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:24.645042 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:24.645202 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:24.645176 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:24.645299 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:24.645246 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:25.796483 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:25.796295 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:03:25.796969 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:25.796853 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"d98d4eada4a92bc46e138081d5ea2cc7ccc51f6a4ea148dd9dd4e78d427ca60e"} Jun 06 22:03:25.799069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:25.799039 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" event={"ID":"460c7697-a783-40cb-a171-2abe6550da31","Type":"ContainerStarted","Data":"87ac7421c4c69a2069742b4600a4254861ed7073f6383cd1f54807ebd07bd0ad"} Jun 06 22:03:25.816781 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:25.816738 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cgmlg" podStartSLOduration=5.062417064 podStartE2EDuration="25.816724695s" totalImagesPullingTime="20.754307631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:25.816704107 +0000 UTC m=+25.565268827" watchObservedRunningTime="2026-06-06 22:03:25.816724695 +0000 UTC m=+25.565289392" Jun 06 22:03:26.644696 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:26.644665 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:26.644855 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:26.644665 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:26.644855 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:26.644791 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:26.644855 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:26.644665 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:26.645011 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:26.644881 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:26.645011 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:26.644927 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:27.805787 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.805611 2559 generic.go:356] "Generic (PLEG): container finished" podID="b24c7544-f928-4f19-ba03-5f51f474f7f8" containerID="b35a9eeda67b2c38b90fd9efe046c22c74e441101b62f1520d96d4c768635617" exitCode=0 Jun 06 22:03:27.806460 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.805701 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerDied","Data":"b35a9eeda67b2c38b90fd9efe046c22c74e441101b62f1520d96d4c768635617"} Jun 06 22:03:27.809324 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.809308 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:03:27.809680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.809661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"a502152c96e7493e05a56c7dbd3cfbf6d8f6d74fae80e4fb8029f59d81ebfdd8"} Jun 06 22:03:27.809935 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.809911 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:27.810027 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.809976 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:27.810130 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.810085 2559 scope.go:117] "RemoveContainer" containerID="0d4823a8c26ac1aa680b508cd4f1cdaec5db2498b0eebfa1057fb6e53da5cd0b" Jun 06 22:03:27.825798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:27.825779 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:28.645230 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.645195 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:28.645360 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.645195 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:28.645360 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:28.645325 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:28.645431 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.645368 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:28.645471 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:28.645453 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:28.645536 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:28.645517 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:28.814439 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.814352 2559 generic.go:356] "Generic (PLEG): container finished" podID="b24c7544-f928-4f19-ba03-5f51f474f7f8" containerID="e0e13fe16ecfcae401ce8b8633a22bbe3a39b01235d5dac4304d6c394edd0e9b" exitCode=0 Jun 06 22:03:28.814889 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.814432 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerDied","Data":"e0e13fe16ecfcae401ce8b8633a22bbe3a39b01235d5dac4304d6c394edd0e9b"} Jun 06 22:03:28.818406 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.818388 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:03:28.818727 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.818708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" event={"ID":"e2fd752e-f7f8-44e5-beaa-5aed253e7e19","Type":"ContainerStarted","Data":"414135e12210662244a21655649f4660a3f2441681e32e7095178702b6515af3"} Jun 06 22:03:28.818967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.818942 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:28.834705 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.834682 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:03:28.875409 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:28.875363 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" podStartSLOduration=10.22134958 podStartE2EDuration="28.875348049s" totalImagesPullingTime="18.653998469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:28.875333469 +0000 UTC m=+28.623898188" watchObservedRunningTime="2026-06-06 22:03:28.875348049 +0000 UTC m=+28.623912749" Jun 06 22:03:29.193781 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.193557 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k6f4v"] Jun 06 22:03:29.193936 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.193800 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:29.193936 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:29.193912 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:29.196537 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.196513 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z46ck"] Jun 06 22:03:29.196655 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.196632 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:29.196733 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:29.196713 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:29.210872 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.210836 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-978qx"] Jun 06 22:03:29.210980 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.210929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:29.211080 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:29.211020 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:29.822811 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.822726 2559 generic.go:356] "Generic (PLEG): container finished" podID="b24c7544-f928-4f19-ba03-5f51f474f7f8" containerID="fd31f849ac1d73a407ccbd74f883048e9780fe2a9360fd4a0225516b697d109a" exitCode=0 Jun 06 22:03:29.822811 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:29.822801 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerDied","Data":"fd31f849ac1d73a407ccbd74f883048e9780fe2a9360fd4a0225516b697d109a"} Jun 06 22:03:30.646053 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:30.646019 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:30.646053 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:30.646046 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:30.646283 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:30.646150 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:30.646283 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:30.646230 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:30.646380 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:30.646336 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:30.646380 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:30.646352 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:32.645204 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:32.645171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:32.645807 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:32.645289 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:32.645922 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:32.645861 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:32.645971 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:32.645944 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:32.646075 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:32.646060 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:32.646185 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:32.646162 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:34.335831 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.335774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:34.336230 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.335916 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:34.336230 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.335979 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:06.335963915 +0000 UTC m=+66.084528616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:34.436924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.436899 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:34.437032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.436946 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:34.437069 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.437028 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:34.437069 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.437065 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:34.437160 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.437079 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:34.437160 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.437083 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret podName:31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:06.437069219 +0000 UTC m=+66.185633916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret") pod "global-pull-secret-syncer-k6f4v" (UID: "31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:34.437160 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.437088 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mxnvb for pod openshift-network-diagnostics/network-check-target-z46ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:34.437160 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.437145 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb podName:3a4f9ed9-924c-4c1d-a928-640982b35606 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:06.437134517 +0000 UTC m=+66.185699214 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxnvb" (UniqueName: "kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb") pod "network-check-target-z46ck" (UID: "3a4f9ed9-924c-4c1d-a928-640982b35606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:34.644670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.644608 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:34.644670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.644621 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:34.644670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.644629 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:34.644895 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.644705 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:03:34.644895 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.644770 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k6f4v" podUID="31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7" Jun 06 22:03:34.644895 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:34.644843 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z46ck" podUID="3a4f9ed9-924c-4c1d-a928-640982b35606" Jun 06 22:03:34.836629 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.836599 2559 generic.go:356] "Generic (PLEG): container finished" podID="b24c7544-f928-4f19-ba03-5f51f474f7f8" containerID="34b519c752acfd088502ab1dbbab677687d8d69ce405d744f9f9a573df11db28" exitCode=0 Jun 06 22:03:34.836629 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:34.836638 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerDied","Data":"34b519c752acfd088502ab1dbbab677687d8d69ce405d744f9f9a573df11db28"} Jun 06 22:03:35.841793 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:35.841766 2559 generic.go:356] "Generic (PLEG): container finished" podID="b24c7544-f928-4f19-ba03-5f51f474f7f8" containerID="4a02dc427eaab8ee13131373e029bd1d47ea2a3238b527db6f66cef3789f1c51" exitCode=0 Jun 06 22:03:35.842162 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:35.841815 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerDied","Data":"4a02dc427eaab8ee13131373e029bd1d47ea2a3238b527db6f66cef3789f1c51"} Jun 06 22:03:36.070839 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.070649 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeReady" Jun 06 22:03:36.070920 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.070871 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 06 22:03:36.112278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.112254 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tmt8x"] Jun 06 22:03:36.126932 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.126911 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rc8nj"] Jun 06 22:03:36.127074 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.127057 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.130250 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.129904 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 06 22:03:36.130250 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.129973 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 06 22:03:36.130250 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.130130 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j597l\"" Jun 06 22:03:36.149729 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.149710 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tmt8x"] Jun 06 22:03:36.149819 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.149742 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rc8nj"] Jun 06 22:03:36.149819 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.149813 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.152265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.152250 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 06 22:03:36.152368 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.152253 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nsrrs\"" Jun 06 22:03:36.152368 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.152333 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 06 22:03:36.152478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.152385 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 06 22:03:36.250299 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.250277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78fb49bc-b80f-4ade-85be-ae550676a352-cert\") pod \"ingress-canary-rc8nj\" (UID: \"78fb49bc-b80f-4ade-85be-ae550676a352\") " pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.250430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.250311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.250430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.250349 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h2lp\" (UniqueName: \"kubernetes.io/projected/78c6ae3d-645e-4097-b9a8-68426888d9f2-kube-api-access-8h2lp\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.250430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.250368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqrfv\" (UniqueName: \"kubernetes.io/projected/78fb49bc-b80f-4ade-85be-ae550676a352-kube-api-access-zqrfv\") pod \"ingress-canary-rc8nj\" (UID: \"78fb49bc-b80f-4ade-85be-ae550676a352\") " pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.250430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.250392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78c6ae3d-645e-4097-b9a8-68426888d9f2-config-volume\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.250430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.250428 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78c6ae3d-645e-4097-b9a8-68426888d9f2-tmp-dir\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.351540 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351468 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.351540 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351513 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h2lp\" (UniqueName: \"kubernetes.io/projected/78c6ae3d-645e-4097-b9a8-68426888d9f2-kube-api-access-8h2lp\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.351540 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351528 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqrfv\" (UniqueName: \"kubernetes.io/projected/78fb49bc-b80f-4ade-85be-ae550676a352-kube-api-access-zqrfv\") pod \"ingress-canary-rc8nj\" (UID: \"78fb49bc-b80f-4ade-85be-ae550676a352\") " pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.351771 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351564 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78c6ae3d-645e-4097-b9a8-68426888d9f2-config-volume\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.351771 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78c6ae3d-645e-4097-b9a8-68426888d9f2-tmp-dir\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.351771 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351623 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78fb49bc-b80f-4ade-85be-ae550676a352-cert\") pod \"ingress-canary-rc8nj\" (UID: \"78fb49bc-b80f-4ade-85be-ae550676a352\") " pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.351771 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:36.351623 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:36.351771 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:36.351720 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:36.851700424 +0000 UTC m=+36.600265127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:03:36.351999 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.351944 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78c6ae3d-645e-4097-b9a8-68426888d9f2-tmp-dir\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.352217 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.352199 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78c6ae3d-645e-4097-b9a8-68426888d9f2-config-volume\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.355996 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.355975 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78fb49bc-b80f-4ade-85be-ae550676a352-cert\") pod \"ingress-canary-rc8nj\" (UID: \"78fb49bc-b80f-4ade-85be-ae550676a352\") " pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.361598 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.361578 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h2lp\" (UniqueName: \"kubernetes.io/projected/78c6ae3d-645e-4097-b9a8-68426888d9f2-kube-api-access-8h2lp\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.361810 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.361788 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqrfv\" (UniqueName: \"kubernetes.io/projected/78fb49bc-b80f-4ade-85be-ae550676a352-kube-api-access-zqrfv\") pod \"ingress-canary-rc8nj\" (UID: \"78fb49bc-b80f-4ade-85be-ae550676a352\") " pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.457855 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.457823 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rc8nj" Jun 06 22:03:36.638959 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.638896 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rc8nj"] Jun 06 22:03:36.642829 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:03:36.642801 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fb49bc_b80f_4ade_85be_ae550676a352.slice/crio-57d7210a952328cd8c5a26da4603c964786f326e94467703308c879d7e4b3ad8 WatchSource:0}: Error finding container 57d7210a952328cd8c5a26da4603c964786f326e94467703308c879d7e4b3ad8: Status 404 returned error can't find the container with id 57d7210a952328cd8c5a26da4603c964786f326e94467703308c879d7e4b3ad8 Jun 06 22:03:36.644512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.644403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:03:36.644512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.644409 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:03:36.644512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.644405 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:03:36.647081 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.647027 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 06 22:03:36.647081 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.647065 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 06 22:03:36.647081 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.647066 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 06 22:03:36.647289 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.647203 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d4tj5\"" Jun 06 22:03:36.647289 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.647240 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 06 22:03:36.647446 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.647339 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ksb8v\"" Jun 06 22:03:36.847226 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.847192 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" event={"ID":"b24c7544-f928-4f19-ba03-5f51f474f7f8","Type":"ContainerStarted","Data":"8407e34ce0d89468a7e398f48565a5adf267748be78085a2de5a45ba99382035"} Jun 06 22:03:36.848303 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.848249 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rc8nj" event={"ID":"78fb49bc-b80f-4ade-85be-ae550676a352","Type":"ContainerStarted","Data":"57d7210a952328cd8c5a26da4603c964786f326e94467703308c879d7e4b3ad8"} Jun 06 22:03:36.856472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.856453 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:36.856593 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:36.856580 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:36.856637 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:36.856632 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:37.85661736 +0000 UTC m=+37.605182062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:03:36.868655 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:36.868615 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qjk7t" podStartSLOduration=8.387779266 podStartE2EDuration="36.868602222s" totalImagesPullingTime="28.480822956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:36.867914781 +0000 UTC m=+36.616479499" watchObservedRunningTime="2026-06-06 22:03:36.868602222 +0000 UTC m=+36.617166974" Jun 06 22:03:37.865699 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:37.865657 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:37.866119 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:37.865825 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:37.866119 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:37.865889 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:39.865874326 +0000 UTC m=+39.614439027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:03:38.854130 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:38.854077 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rc8nj" event={"ID":"78fb49bc-b80f-4ade-85be-ae550676a352","Type":"ContainerStarted","Data":"8e7dd17a3c27cb600411261467b1e1808b781e76367438a62f3a51d9f0f2276b"} Jun 06 22:03:38.869413 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:38.869360 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rc8nj" podStartSLOduration=1.071916342 podStartE2EDuration="2.869343312s" totalImagesPullingTime="1.79742697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:38.869083684 +0000 UTC m=+38.617648402" watchObservedRunningTime="2026-06-06 22:03:38.869343312 +0000 UTC m=+38.617908011" Jun 06 22:03:39.879543 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:39.879513 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:39.879904 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:39.879616 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:39.879904 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:39.879663 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:43.879649396 +0000 UTC m=+43.628214112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:03:43.909686 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:43.909653 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:43.910043 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:43.909819 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:43.910043 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:43.909881 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:51.909863664 +0000 UTC m=+51.658428361 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:03:51.965973 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:03:51.965937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:03:51.966470 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:51.966080 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:51.966470 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:03:51.966164 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:07.966146737 +0000 UTC m=+67.714711434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:04:00.835548 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:00.835521 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dxdx6" Jun 06 22:04:06.367013 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.366971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:04:06.369544 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.369525 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 06 22:04:06.377547 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:04:06.377525 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 06 22:04:06.377673 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:04:06.377594 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:05:10.377576026 +0000 UTC m=+130.126140723 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : secret "metrics-daemon-secret" not found Jun 06 22:04:06.467311 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.467278 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:04:06.467470 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.467328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:04:06.469914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.469895 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 06 22:04:06.470018 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.469896 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 06 22:04:06.480085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.480060 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 06 22:04:06.484266 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.484247 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7-original-pull-secret\") pod \"global-pull-secret-syncer-k6f4v\" (UID: \"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7\") " pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:04:06.490374 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.490352 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxnvb\" (UniqueName: \"kubernetes.io/projected/3a4f9ed9-924c-4c1d-a928-640982b35606-kube-api-access-mxnvb\") pod \"network-check-target-z46ck\" (UID: \"3a4f9ed9-924c-4c1d-a928-640982b35606\") " pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:04:06.664176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.664053 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ksb8v\"" Jun 06 22:04:06.665072 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.665058 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k6f4v" Jun 06 22:04:06.672602 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.672570 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:04:06.793376 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.793346 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k6f4v"] Jun 06 22:04:06.796051 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:04:06.796022 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c22d88_e0a9_418a_8b27_b5fe5f2d1ef7.slice/crio-eb96b8f2cd82e94205bc8b9c8d1f438835ef6d7d609d531fa5e041e3be0fd324 WatchSource:0}: Error finding container eb96b8f2cd82e94205bc8b9c8d1f438835ef6d7d609d531fa5e041e3be0fd324: Status 404 returned error can't find the container with id eb96b8f2cd82e94205bc8b9c8d1f438835ef6d7d609d531fa5e041e3be0fd324 Jun 06 22:04:06.809684 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.809660 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z46ck"] Jun 06 22:04:06.812817 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:04:06.812794 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4f9ed9_924c_4c1d_a928_640982b35606.slice/crio-3b517291e43add82a22be0fa01a3377e83d3ee5d73f2080a5248a0f9ac5327e4 WatchSource:0}: Error finding container 3b517291e43add82a22be0fa01a3377e83d3ee5d73f2080a5248a0f9ac5327e4: Status 404 returned error can't find the container with id 3b517291e43add82a22be0fa01a3377e83d3ee5d73f2080a5248a0f9ac5327e4 Jun 06 22:04:06.917565 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.917469 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k6f4v" event={"ID":"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7","Type":"ContainerStarted","Data":"eb96b8f2cd82e94205bc8b9c8d1f438835ef6d7d609d531fa5e041e3be0fd324"} Jun 06 22:04:06.918535 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:06.918513 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z46ck" event={"ID":"3a4f9ed9-924c-4c1d-a928-640982b35606","Type":"ContainerStarted","Data":"3b517291e43add82a22be0fa01a3377e83d3ee5d73f2080a5248a0f9ac5327e4"} Jun 06 22:04:07.979951 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:07.979923 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:04:07.980458 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:04:07.980088 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:04:07.980458 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:04:07.980173 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:39.980152486 +0000 UTC m=+99.728717202 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:04:11.931693 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:11.931647 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k6f4v" event={"ID":"31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7","Type":"ContainerStarted","Data":"7f0802502308c2843cba36e7774df4a765db497418b0a4232ceea7e3eb630de9"} Jun 06 22:04:11.933072 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:11.933047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z46ck" event={"ID":"3a4f9ed9-924c-4c1d-a928-640982b35606","Type":"ContainerStarted","Data":"6a4531c2ed7791b98650a24bae205098d285c65d53d036e7e395755e250f8416"} Jun 06 22:04:11.933198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:11.933140 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:04:11.946449 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:11.946404 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k6f4v" podStartSLOduration=67.702552697 podStartE2EDuration="1m11.946390421s" totalImagesPullingTime="4.243837724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:04:11.94576957 +0000 UTC m=+71.694334290" watchObservedRunningTime="2026-06-06 22:04:11.946390421 +0000 UTC m=+71.694955139" Jun 06 22:04:11.959924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:11.959880 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z46ck" podStartSLOduration=67.738423269 podStartE2EDuration="1m11.959865148s" totalImagesPullingTime="4.221441879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:04:11.959164258 +0000 UTC m=+71.707728976" watchObservedRunningTime="2026-06-06 22:04:11.959865148 +0000 UTC m=+71.708429867" Jun 06 22:04:39.998337 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:39.998291 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:04:39.998758 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:04:39.998400 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:04:39.998758 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:04:39.998493 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:05:43.99847385 +0000 UTC m=+163.747038551 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:04:42.937491 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:04:42.937460 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z46ck" Jun 06 22:05:10.388015 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:05:10.387968 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:05:10.388548 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:05:10.388085 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 06 22:05:10.388548 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:05:10.388154 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs podName:7635dc64-f99d-4422-99ff-419efdcefc01 nodeName:}" failed. No retries permitted until 2026-06-06 22:07:12.388139878 +0000 UTC m=+252.136704575 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs") pod "network-metrics-daemon-978qx" (UID: "7635dc64-f99d-4422-99ff-419efdcefc01") : secret "metrics-daemon-secret" not found Jun 06 22:05:39.135996 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:05:39.135912 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tmt8x" podUID="78c6ae3d-645e-4097-b9a8-68426888d9f2" Jun 06 22:05:39.655132 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:05:39.655064 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-978qx" podUID="7635dc64-f99d-4422-99ff-419efdcefc01" Jun 06 22:05:40.138636 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:05:40.138610 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tmt8x" Jun 06 22:05:44.020302 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:05:44.020235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:05:44.020702 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:05:44.020397 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:05:44.020702 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:05:44.020473 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls podName:78c6ae3d-645e-4097-b9a8-68426888d9f2 nodeName:}" failed. No retries permitted until 2026-06-06 22:07:46.020457014 +0000 UTC m=+285.769021709 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls") pod "dns-default-tmt8x" (UID: "78c6ae3d-645e-4097-b9a8-68426888d9f2") : secret "dns-default-metrics-tls" not found Jun 06 22:05:53.644894 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:05:53.644800 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:06:13.956882 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.956850 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-hpmgc"] Jun 06 22:06:13.959265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.959248 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv"] Jun 06 22:06:13.959379 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.959362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:13.961542 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.961525 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d8fb8b8-h4jf7"] Jun 06 22:06:13.961660 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.961587 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 06 22:06:13.961660 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.961647 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:13.961926 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.961900 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 06 22:06:13.962149 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.962134 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 06 22:06:13.962389 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.962376 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5d86q\"" Jun 06 22:06:13.962694 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.962675 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 06 22:06:13.963950 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.963931 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht"] Jun 06 22:06:13.964040 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.964028 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:13.964786 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.964769 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jxnzn\"" Jun 06 22:06:13.966517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966131 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 06 22:06:13.966517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966414 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" Jun 06 22:06:13.966874 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966852 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 06 22:06:13.966967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966889 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 06 22:06:13.966967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966902 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 06 22:06:13.966967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966929 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 06 22:06:13.967161 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.966860 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 06 22:06:13.967322 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.967308 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zzm52\"" Jun 06 22:06:13.967452 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.967439 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 06 22:06:13.968674 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.968657 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 06 22:06:13.971368 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.971347 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 06 22:06:13.973573 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.973549 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-w2jn7\"" Jun 06 22:06:13.977361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.977343 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv"] Jun 06 22:06:13.983642 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.983618 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-hpmgc"] Jun 06 22:06:13.984271 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.984253 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht"] Jun 06 22:06:13.986156 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:13.986136 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d8fb8b8-h4jf7"] Jun 06 22:06:14.018611 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.018611 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aa2f6a1-8045-4da3-aad4-588972d9d972-ca-trust-extracted\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018624 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-installation-pull-secrets\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018642 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wnm6\" (UniqueName: \"kubernetes.io/projected/e36d27da-a653-45a8-b223-a2f2118773ca-kube-api-access-9wnm6\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018661 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8f6cde2-e582-46f0-9d75-bbe9327855cb-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6r4s\" (UniqueName: \"kubernetes.io/projected/642681aa-3d97-4c37-98ea-4cf79982a6c0-kube-api-access-p6r4s\") pod \"network-check-source-dbd9cf844-pgkht\" (UID: \"642681aa-3d97-4c37-98ea-4cf79982a6c0\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018727 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e36d27da-a653-45a8-b223-a2f2118773ca-service-ca-bundle\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018787 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-image-registry-private-configuration\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.018830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e36d27da-a653-45a8-b223-a2f2118773ca-serving-cert\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-bound-sa-token\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018871 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e36d27da-a653-45a8-b223-a2f2118773ca-tmp\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e36d27da-a653-45a8-b223-a2f2118773ca-snapshots\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvjl4\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-kube-api-access-kvjl4\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018952 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e36d27da-a653-45a8-b223-a2f2118773ca-trusted-ca-bundle\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5k48\" (UniqueName: \"kubernetes.io/projected/e8f6cde2-e582-46f0-9d75-bbe9327855cb-kube-api-access-m5k48\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.018986 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-certificates\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.019045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.019004 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-trusted-ca\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.019309 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.019054 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.120350 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120313 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120366 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aa2f6a1-8045-4da3-aad4-588972d9d972-ca-trust-extracted\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120400 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-installation-pull-secrets\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120427 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wnm6\" (UniqueName: \"kubernetes.io/projected/e36d27da-a653-45a8-b223-a2f2118773ca-kube-api-access-9wnm6\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120456 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8f6cde2-e582-46f0-9d75-bbe9327855cb-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.120472 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.120494 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d8fb8b8-h4jf7: secret "image-registry-tls" not found Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.120497 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120480 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6r4s\" (UniqueName: \"kubernetes.io/projected/642681aa-3d97-4c37-98ea-4cf79982a6c0-kube-api-access-p6r4s\") pod \"network-check-source-dbd9cf844-pgkht\" (UID: \"642681aa-3d97-4c37-98ea-4cf79982a6c0\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" Jun 06 22:06:14.120557 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.120559 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls podName:4aa2f6a1-8045-4da3-aad4-588972d9d972 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:14.620538409 +0000 UTC m=+194.369103127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls") pod "image-registry-6d8fb8b8-h4jf7" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972") : secret "image-registry-tls" not found Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.120655 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls podName:e8f6cde2-e582-46f0-9d75-bbe9327855cb nodeName:}" failed. No retries permitted until 2026-06-06 22:06:14.620638554 +0000 UTC m=+194.369203255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kcnzv" (UID: "e8f6cde2-e582-46f0-9d75-bbe9327855cb") : secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120679 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e36d27da-a653-45a8-b223-a2f2118773ca-service-ca-bundle\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-image-registry-private-configuration\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e36d27da-a653-45a8-b223-a2f2118773ca-serving-cert\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120798 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-bound-sa-token\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120804 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aa2f6a1-8045-4da3-aad4-588972d9d972-ca-trust-extracted\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e36d27da-a653-45a8-b223-a2f2118773ca-tmp\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120855 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e36d27da-a653-45a8-b223-a2f2118773ca-snapshots\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120897 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvjl4\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-kube-api-access-kvjl4\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e36d27da-a653-45a8-b223-a2f2118773ca-trusted-ca-bundle\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5k48\" (UniqueName: \"kubernetes.io/projected/e8f6cde2-e582-46f0-9d75-bbe9327855cb-kube-api-access-m5k48\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.120977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-certificates\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.121020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-trusted-ca\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.121832 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121146 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e36d27da-a653-45a8-b223-a2f2118773ca-tmp\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121832 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121305 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e36d27da-a653-45a8-b223-a2f2118773ca-service-ca-bundle\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.121832 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121346 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8f6cde2-e582-46f0-9d75-bbe9327855cb-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.121832 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121654 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e36d27da-a653-45a8-b223-a2f2118773ca-snapshots\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.122091 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121931 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-certificates\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.122091 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.121962 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e36d27da-a653-45a8-b223-a2f2118773ca-trusted-ca-bundle\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.122239 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.122180 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-trusted-ca\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.123405 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.123377 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e36d27da-a653-45a8-b223-a2f2118773ca-serving-cert\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.123519 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.123448 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-image-registry-private-configuration\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.123752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.123735 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-installation-pull-secrets\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.135418 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.135396 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-bound-sa-token\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.135569 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.135544 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5k48\" (UniqueName: \"kubernetes.io/projected/e8f6cde2-e582-46f0-9d75-bbe9327855cb-kube-api-access-m5k48\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.135665 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.135554 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvjl4\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-kube-api-access-kvjl4\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.136029 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.136011 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wnm6\" (UniqueName: \"kubernetes.io/projected/e36d27da-a653-45a8-b223-a2f2118773ca-kube-api-access-9wnm6\") pod \"insights-operator-896549976-hpmgc\" (UID: \"e36d27da-a653-45a8-b223-a2f2118773ca\") " pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.136782 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.136759 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6r4s\" (UniqueName: \"kubernetes.io/projected/642681aa-3d97-4c37-98ea-4cf79982a6c0-kube-api-access-p6r4s\") pod \"network-check-source-dbd9cf844-pgkht\" (UID: \"642681aa-3d97-4c37-98ea-4cf79982a6c0\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" Jun 06 22:06:14.149726 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.149702 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk"] Jun 06 22:06:14.153310 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.153296 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.156451 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.156429 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 06 22:06:14.156660 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.156647 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 06 22:06:14.156753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.156685 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 06 22:06:14.156811 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.156790 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hwbjj\"" Jun 06 22:06:14.156865 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.156816 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:06:14.163114 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.163072 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk"] Jun 06 22:06:14.222202 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.222123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlbz5\" (UniqueName: \"kubernetes.io/projected/48b6c9d8-995f-499c-8695-640e911922c7-kube-api-access-qlbz5\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.222202 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.222155 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/48b6c9d8-995f-499c-8695-640e911922c7-config\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.222202 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.222185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48b6c9d8-995f-499c-8695-640e911922c7-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.270082 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.270052 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-hpmgc" Jun 06 22:06:14.290062 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.290039 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" Jun 06 22:06:14.322864 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.322830 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/48b6c9d8-995f-499c-8695-640e911922c7-config\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.322997 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.322900 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48b6c9d8-995f-499c-8695-640e911922c7-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.323155 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.323127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlbz5\" (UniqueName: \"kubernetes.io/projected/48b6c9d8-995f-499c-8695-640e911922c7-kube-api-access-qlbz5\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.323694 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.323641 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/48b6c9d8-995f-499c-8695-640e911922c7-config\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.325073 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.325048 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48b6c9d8-995f-499c-8695-640e911922c7-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.331498 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.331461 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlbz5\" (UniqueName: \"kubernetes.io/projected/48b6c9d8-995f-499c-8695-640e911922c7-kube-api-access-qlbz5\") pod \"service-ca-operator-6bc5ff68f5-lmbjk\" (UID: \"48b6c9d8-995f-499c-8695-640e911922c7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.401261 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.401217 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-hpmgc"] Jun 06 22:06:14.404839 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:14.404814 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36d27da_a653_45a8_b223_a2f2118773ca.slice/crio-2f6c7c55c22d234ac90837459edfa1d06342cc0c6213708e213bc20617119478 WatchSource:0}: Error finding container 2f6c7c55c22d234ac90837459edfa1d06342cc0c6213708e213bc20617119478: Status 404 returned error can't find the container with id 2f6c7c55c22d234ac90837459edfa1d06342cc0c6213708e213bc20617119478 Jun 06 22:06:14.416536 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.416511 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht"] Jun 06 22:06:14.418752 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:14.418731 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642681aa_3d97_4c37_98ea_4cf79982a6c0.slice/crio-c0b1c1d5d1b7c6b6828ca8327e20fb3f4d5fd871d3effa15f8ca00dba97cf335 WatchSource:0}: Error finding container c0b1c1d5d1b7c6b6828ca8327e20fb3f4d5fd871d3effa15f8ca00dba97cf335: Status 404 returned error can't find the container with id c0b1c1d5d1b7c6b6828ca8327e20fb3f4d5fd871d3effa15f8ca00dba97cf335 Jun 06 22:06:14.461410 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.461389 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" Jun 06 22:06:14.581786 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.581757 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk"] Jun 06 22:06:14.584664 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:14.584622 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b6c9d8_995f_499c_8695_640e911922c7.slice/crio-4d9744977c7e3971bb4fad2452e1c3c4665141a00c0ff856e844f91264110d63 WatchSource:0}: Error finding container 4d9744977c7e3971bb4fad2452e1c3c4665141a00c0ff856e844f91264110d63: Status 404 returned error can't find the container with id 4d9744977c7e3971bb4fad2452e1c3c4665141a00c0ff856e844f91264110d63 Jun 06 22:06:14.625783 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.625744 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:14.625938 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:14.625802 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:14.625938 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.625912 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 06 22:06:14.625938 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.625934 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d8fb8b8-h4jf7: secret "image-registry-tls" not found Jun 06 22:06:14.626091 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.625949 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:14.626091 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.625992 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls podName:4aa2f6a1-8045-4da3-aad4-588972d9d972 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:15.625975128 +0000 UTC m=+195.374539827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls") pod "image-registry-6d8fb8b8-h4jf7" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972") : secret "image-registry-tls" not found Jun 06 22:06:14.626091 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:14.626008 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls podName:e8f6cde2-e582-46f0-9d75-bbe9327855cb nodeName:}" failed. No retries permitted until 2026-06-06 22:06:15.626000201 +0000 UTC m=+195.374564897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kcnzv" (UID: "e8f6cde2-e582-46f0-9d75-bbe9327855cb") : secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:15.220706 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.220666 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" event={"ID":"48b6c9d8-995f-499c-8695-640e911922c7","Type":"ContainerStarted","Data":"4d9744977c7e3971bb4fad2452e1c3c4665141a00c0ff856e844f91264110d63"} Jun 06 22:06:15.222619 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.222589 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" event={"ID":"642681aa-3d97-4c37-98ea-4cf79982a6c0","Type":"ContainerStarted","Data":"9b6a8244a38d39a3c13c6dd9f68f2efd358564f239ee4b5138240f7db20e93d1"} Jun 06 22:06:15.222726 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.222638 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" event={"ID":"642681aa-3d97-4c37-98ea-4cf79982a6c0","Type":"ContainerStarted","Data":"c0b1c1d5d1b7c6b6828ca8327e20fb3f4d5fd871d3effa15f8ca00dba97cf335"} Jun 06 22:06:15.224379 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.224353 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hpmgc" event={"ID":"e36d27da-a653-45a8-b223-a2f2118773ca","Type":"ContainerStarted","Data":"2f6c7c55c22d234ac90837459edfa1d06342cc0c6213708e213bc20617119478"} Jun 06 22:06:15.238491 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.238431 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-pgkht" podStartSLOduration=2.238410369 podStartE2EDuration="2.238410369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:15.23805352 +0000 UTC m=+194.986618241" watchObservedRunningTime="2026-06-06 22:06:15.238410369 +0000 UTC m=+194.986975090" Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.636069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:15.636148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:15.636230 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:15.636254 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d8fb8b8-h4jf7: secret "image-registry-tls" not found Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:15.636293 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:15.636317 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls podName:4aa2f6a1-8045-4da3-aad4-588972d9d972 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:17.636296003 +0000 UTC m=+197.384860723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls") pod "image-registry-6d8fb8b8-h4jf7" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972") : secret "image-registry-tls" not found Jun 06 22:06:15.636524 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:15.636337 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls podName:e8f6cde2-e582-46f0-9d75-bbe9327855cb nodeName:}" failed. No retries permitted until 2026-06-06 22:06:17.636325062 +0000 UTC m=+197.384889760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kcnzv" (UID: "e8f6cde2-e582-46f0-9d75-bbe9327855cb") : secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:17.232923 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:17.232877 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hpmgc" event={"ID":"e36d27da-a653-45a8-b223-a2f2118773ca","Type":"ContainerStarted","Data":"ed90234fe1d4bc5657a0d067fa16585bb57a67a9044f86e4d2b5a2b0901c6d59"} Jun 06 22:06:17.234358 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:17.234328 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" event={"ID":"48b6c9d8-995f-499c-8695-640e911922c7","Type":"ContainerStarted","Data":"c7557b1f45b2640549d75920a292dda56e97da145cfbaa89576aa89510755de6"} Jun 06 22:06:17.251567 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:17.251523 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-hpmgc" podStartSLOduration=1.609962188 podStartE2EDuration="4.251509225s" totalImagesPullingTime="2.641547037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:17.250255254 +0000 UTC m=+196.998819975" watchObservedRunningTime="2026-06-06 22:06:17.251509225 +0000 UTC m=+197.000073943" Jun 06 22:06:17.265235 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:17.265189 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" podStartSLOduration=0.80436413 podStartE2EDuration="3.265176616s" totalImagesPullingTime="2.460812486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:17.264629901 +0000 UTC m=+197.013194633" watchObservedRunningTime="2026-06-06 22:06:17.265176616 +0000 UTC m=+197.013741335" Jun 06 22:06:17.654205 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:17.654174 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:17.654391 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:17.654213 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:17.654391 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:17.654329 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:17.654391 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:17.654334 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 06 22:06:17.654391 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:17.654352 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d8fb8b8-h4jf7: secret "image-registry-tls" not found Jun 06 22:06:17.654597 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:17.654399 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls podName:e8f6cde2-e582-46f0-9d75-bbe9327855cb nodeName:}" failed. No retries permitted until 2026-06-06 22:06:21.654380656 +0000 UTC m=+201.402945378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kcnzv" (UID: "e8f6cde2-e582-46f0-9d75-bbe9327855cb") : secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:17.654597 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:17.654416 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls podName:4aa2f6a1-8045-4da3-aad4-588972d9d972 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:21.654408156 +0000 UTC m=+201.402972852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls") pod "image-registry-6d8fb8b8-h4jf7" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972") : secret "image-registry-tls" not found Jun 06 22:06:19.086317 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.086284 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm"] Jun 06 22:06:19.089473 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.089458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" Jun 06 22:06:19.093037 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.093017 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bmd7l\"" Jun 06 22:06:19.093188 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.093018 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 06 22:06:19.093548 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.093530 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 06 22:06:19.105158 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.105129 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm"] Jun 06 22:06:19.166784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.166752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk6np\" (UniqueName: \"kubernetes.io/projected/b4429180-f6f8-4e31-bb06-d953bbe832fe-kube-api-access-nk6np\") pod \"migrator-955ffdf5f-nfckm\" (UID: \"b4429180-f6f8-4e31-bb06-d953bbe832fe\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" Jun 06 22:06:19.268064 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.268029 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk6np\" (UniqueName: \"kubernetes.io/projected/b4429180-f6f8-4e31-bb06-d953bbe832fe-kube-api-access-nk6np\") pod \"migrator-955ffdf5f-nfckm\" (UID: \"b4429180-f6f8-4e31-bb06-d953bbe832fe\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" Jun 06 22:06:19.276593 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.276571 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk6np\" (UniqueName: \"kubernetes.io/projected/b4429180-f6f8-4e31-bb06-d953bbe832fe-kube-api-access-nk6np\") pod \"migrator-955ffdf5f-nfckm\" (UID: \"b4429180-f6f8-4e31-bb06-d953bbe832fe\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" Jun 06 22:06:19.397685 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.397595 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" Jun 06 22:06:19.523591 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.523566 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm"] Jun 06 22:06:19.525882 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:19.525848 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4429180_f6f8_4e31_bb06_d953bbe832fe.slice/crio-4ab581946da4554669c6c88d5fd53c8e82a86146acfb22c5aff6796be8311c5f WatchSource:0}: Error finding container 4ab581946da4554669c6c88d5fd53c8e82a86146acfb22c5aff6796be8311c5f: Status 404 returned error can't find the container with id 4ab581946da4554669c6c88d5fd53c8e82a86146acfb22c5aff6796be8311c5f Jun 06 22:06:19.677292 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:19.677208 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5v4lt_bb57a9c4-fe3e-4f7f-b452-011eee4d5173/dns-node-resolver/0.log" Jun 06 22:06:20.248206 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:20.248170 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" event={"ID":"b4429180-f6f8-4e31-bb06-d953bbe832fe","Type":"ContainerStarted","Data":"4ab581946da4554669c6c88d5fd53c8e82a86146acfb22c5aff6796be8311c5f"} Jun 06 22:06:21.076614 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.076544 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xh4kn_c365b91d-ec96-452a-a635-fffdaf8f6f2c/node-ca/0.log" Jun 06 22:06:21.252809 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.252772 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" event={"ID":"b4429180-f6f8-4e31-bb06-d953bbe832fe","Type":"ContainerStarted","Data":"73b859730d75eba058b63bc3d8f731c2665441259d962eeaa5c1e9dccf2b7407"} Jun 06 22:06:21.253212 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.252829 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" event={"ID":"b4429180-f6f8-4e31-bb06-d953bbe832fe","Type":"ContainerStarted","Data":"4eaa381de6750afe7498221d30b5f443f37e16e8181b748e2a7dba9e53131506"} Jun 06 22:06:21.269236 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.269186 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-nfckm" podStartSLOduration=1.066909855 podStartE2EDuration="2.269169808s" totalImagesPullingTime="1.202259953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:21.26798235 +0000 UTC m=+201.016547069" watchObservedRunningTime="2026-06-06 22:06:21.269169808 +0000 UTC m=+201.017734527" Jun 06 22:06:21.687344 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.687301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:21.687536 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.687354 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:21.687536 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:21.687452 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 06 22:06:21.687536 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:21.687469 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:21.687536 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:21.687472 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d8fb8b8-h4jf7: secret "image-registry-tls" not found Jun 06 22:06:21.687536 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:21.687530 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls podName:4aa2f6a1-8045-4da3-aad4-588972d9d972 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:29.687512432 +0000 UTC m=+209.436077130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls") pod "image-registry-6d8fb8b8-h4jf7" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972") : secret "image-registry-tls" not found Jun 06 22:06:21.687733 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:21.687543 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls podName:e8f6cde2-e582-46f0-9d75-bbe9327855cb nodeName:}" failed. No retries permitted until 2026-06-06 22:06:29.687537257 +0000 UTC m=+209.436101954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kcnzv" (UID: "e8f6cde2-e582-46f0-9d75-bbe9327855cb") : secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:21.876601 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:21.876575 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rc8nj_78fb49bc-b80f-4ade-85be-ae550676a352/serve-healthcheck-canary/0.log" Jun 06 22:06:29.747538 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:29.747498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:29.747538 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:29.747545 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:29.747966 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:29.747677 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:29.747966 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:29.747731 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls podName:e8f6cde2-e582-46f0-9d75-bbe9327855cb nodeName:}" failed. No retries permitted until 2026-06-06 22:06:45.747717876 +0000 UTC m=+225.496282583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kcnzv" (UID: "e8f6cde2-e582-46f0-9d75-bbe9327855cb") : secret "cluster-monitoring-operator-tls" not found Jun 06 22:06:29.750143 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:29.750118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"image-registry-6d8fb8b8-h4jf7\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:29.883764 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:29.883725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:30.010286 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:30.010211 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d8fb8b8-h4jf7"] Jun 06 22:06:30.013126 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:30.013081 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa2f6a1_8045_4da3_aad4_588972d9d972.slice/crio-50259f0ea541e316429e3a7621c037099b5c2a38e61fe68c5361a80a99573a43 WatchSource:0}: Error finding container 50259f0ea541e316429e3a7621c037099b5c2a38e61fe68c5361a80a99573a43: Status 404 returned error can't find the container with id 50259f0ea541e316429e3a7621c037099b5c2a38e61fe68c5361a80a99573a43 Jun 06 22:06:30.285969 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:30.285884 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" event={"ID":"4aa2f6a1-8045-4da3-aad4-588972d9d972","Type":"ContainerStarted","Data":"a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1"} Jun 06 22:06:30.285969 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:30.285934 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" event={"ID":"4aa2f6a1-8045-4da3-aad4-588972d9d972","Type":"ContainerStarted","Data":"50259f0ea541e316429e3a7621c037099b5c2a38e61fe68c5361a80a99573a43"} Jun 06 22:06:30.286178 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:30.285982 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:06:30.305269 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:30.305213 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" podStartSLOduration=17.305197598 podStartE2EDuration="17.305197598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:30.304344811 +0000 UTC m=+210.052909543" watchObservedRunningTime="2026-06-06 22:06:30.305197598 +0000 UTC m=+210.053762320" Jun 06 22:06:39.769589 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.769555 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qknqk"] Jun 06 22:06:39.774805 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.774788 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.777991 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.777968 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vjws4\"" Jun 06 22:06:39.778313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.778292 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 06 22:06:39.779862 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.779841 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 06 22:06:39.790609 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.790586 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qknqk"] Jun 06 22:06:39.829785 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.829754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-crio-socket\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.829943 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.829799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.829943 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.829831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsggc\" (UniqueName: \"kubernetes.io/projected/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-kube-api-access-rsggc\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.829943 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.829880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-data-volume\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.830068 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.829940 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.876387 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.876347 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d8fb8b8-h4jf7"] Jun 06 22:06:39.904321 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.904289 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9b55d5d4c-fghf8"] Jun 06 22:06:39.907657 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.907637 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:39.924298 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.924272 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9b55d5d4c-fghf8"] Jun 06 22:06:39.930467 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.930441 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.930571 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.930493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-crio-socket\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.930571 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.930531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.930571 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.930560 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsggc\" (UniqueName: \"kubernetes.io/projected/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-kube-api-access-rsggc\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.930782 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.930766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-crio-socket\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.930864 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.930843 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-data-volume\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.931141 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.931123 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.931187 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.931132 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-data-volume\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.932860 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.932844 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:39.947435 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:39.947408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsggc\" (UniqueName: \"kubernetes.io/projected/5b0c63a6-59a9-45ff-9f88-a46568a5db8f-kube-api-access-rsggc\") pod \"insights-runtime-extractor-qknqk\" (UID: \"5b0c63a6-59a9-45ff-9f88-a46568a5db8f\") " pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:40.032032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.031944 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aeb90dca-a413-4e71-a86c-49236305ea78-trusted-ca\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.031976 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aeb90dca-a413-4e71-a86c-49236305ea78-image-registry-private-configuration\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.031999 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aeb90dca-a413-4e71-a86c-49236305ea78-ca-trust-extracted\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.032017 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aeb90dca-a413-4e71-a86c-49236305ea78-registry-certificates\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.032079 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aeb90dca-a413-4e71-a86c-49236305ea78-installation-pull-secrets\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.032125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8gnv\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-kube-api-access-w8gnv\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.032146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-bound-sa-token\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.032313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.032183 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-registry-tls\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.083265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.083239 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qknqk" Jun 06 22:06:40.132948 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.132906 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aeb90dca-a413-4e71-a86c-49236305ea78-trusted-ca\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.132952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aeb90dca-a413-4e71-a86c-49236305ea78-image-registry-private-configuration\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.132985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aeb90dca-a413-4e71-a86c-49236305ea78-ca-trust-extracted\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.133010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aeb90dca-a413-4e71-a86c-49236305ea78-registry-certificates\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.133065 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aeb90dca-a413-4e71-a86c-49236305ea78-installation-pull-secrets\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.133118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8gnv\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-kube-api-access-w8gnv\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.133174 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-bound-sa-token\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.133245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.133238 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-registry-tls\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.134388 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.134131 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aeb90dca-a413-4e71-a86c-49236305ea78-ca-trust-extracted\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.134496 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.134467 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aeb90dca-a413-4e71-a86c-49236305ea78-trusted-ca\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.134701 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.134666 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aeb90dca-a413-4e71-a86c-49236305ea78-registry-certificates\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.137081 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.137057 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-registry-tls\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.137376 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.137354 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aeb90dca-a413-4e71-a86c-49236305ea78-image-registry-private-configuration\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.137849 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.137830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aeb90dca-a413-4e71-a86c-49236305ea78-installation-pull-secrets\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.142233 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.142175 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-bound-sa-token\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.142233 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.142185 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8gnv\" (UniqueName: \"kubernetes.io/projected/aeb90dca-a413-4e71-a86c-49236305ea78-kube-api-access-w8gnv\") pod \"image-registry-9b55d5d4c-fghf8\" (UID: \"aeb90dca-a413-4e71-a86c-49236305ea78\") " pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.217659 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.217612 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:40.229849 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.229823 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qknqk"] Jun 06 22:06:40.232428 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:40.232402 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0c63a6_59a9_45ff_9f88_a46568a5db8f.slice/crio-bfc4755c13ae430dfde85b9ef2755514794b60910512b69652e5248e7c3c5f88 WatchSource:0}: Error finding container bfc4755c13ae430dfde85b9ef2755514794b60910512b69652e5248e7c3c5f88: Status 404 returned error can't find the container with id bfc4755c13ae430dfde85b9ef2755514794b60910512b69652e5248e7c3c5f88 Jun 06 22:06:40.325898 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.325864 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknqk" event={"ID":"5b0c63a6-59a9-45ff-9f88-a46568a5db8f","Type":"ContainerStarted","Data":"156d2364a3d9173230ef7ad8496126e1231e2f9e8f1be89b82a65bc2c8f85719"} Jun 06 22:06:40.326045 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.325946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknqk" event={"ID":"5b0c63a6-59a9-45ff-9f88-a46568a5db8f","Type":"ContainerStarted","Data":"bfc4755c13ae430dfde85b9ef2755514794b60910512b69652e5248e7c3c5f88"} Jun 06 22:06:40.389224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:40.389197 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9b55d5d4c-fghf8"] Jun 06 22:06:40.391429 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:40.391395 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb90dca_a413_4e71_a86c_49236305ea78.slice/crio-7816d977757eb0e44b70c58de50969e4f090c8e4717cf207bb5c28c786096326 WatchSource:0}: Error finding container 7816d977757eb0e44b70c58de50969e4f090c8e4717cf207bb5c28c786096326: Status 404 returned error can't find the container with id 7816d977757eb0e44b70c58de50969e4f090c8e4717cf207bb5c28c786096326 Jun 06 22:06:41.332142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:41.332081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknqk" event={"ID":"5b0c63a6-59a9-45ff-9f88-a46568a5db8f","Type":"ContainerStarted","Data":"d1a287534f7ab26311eff15536443fd467ae6f6067c17895b6325cc68db6447d"} Jun 06 22:06:41.333856 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:41.333826 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" event={"ID":"aeb90dca-a413-4e71-a86c-49236305ea78","Type":"ContainerStarted","Data":"16d1b6e2e931b11e110a7f5e9703c9ab131e178daf139e8108e452fcb85bae9a"} Jun 06 22:06:41.333972 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:41.333876 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" event={"ID":"aeb90dca-a413-4e71-a86c-49236305ea78","Type":"ContainerStarted","Data":"7816d977757eb0e44b70c58de50969e4f090c8e4717cf207bb5c28c786096326"} Jun 06 22:06:41.333972 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:41.333957 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:06:41.352337 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:41.352282 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podStartSLOduration=2.35226427 podStartE2EDuration="2.35226427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:41.352065067 +0000 UTC m=+221.100629814" watchObservedRunningTime="2026-06-06 22:06:41.35226427 +0000 UTC m=+221.100828988" Jun 06 22:06:42.339151 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:42.339117 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknqk" event={"ID":"5b0c63a6-59a9-45ff-9f88-a46568a5db8f","Type":"ContainerStarted","Data":"aa21c5d23ae0c421c03a50a58733ac126b4f8160edd92aa880ef102921924215"} Jun 06 22:06:42.360959 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:42.360903 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qknqk" podStartSLOduration=1.449642192 podStartE2EDuration="3.360885678s" totalImagesPullingTime="1.911243486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:06:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:42.359017459 +0000 UTC m=+222.107582179" watchObservedRunningTime="2026-06-06 22:06:42.360885678 +0000 UTC m=+222.109450396" Jun 06 22:06:45.782832 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:45.782797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:45.785302 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:45.785278 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8f6cde2-e582-46f0-9d75-bbe9327855cb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kcnzv\" (UID: \"e8f6cde2-e582-46f0-9d75-bbe9327855cb\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:46.076443 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:46.076384 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" Jun 06 22:06:46.195385 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:46.195355 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv"] Jun 06 22:06:46.198367 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:46.198338 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f6cde2_e582_46f0_9d75_bbe9327855cb.slice/crio-61e00171a9c389ea6388024f256ad2d435da40bb370bf39e96a86d577dd36773 WatchSource:0}: Error finding container 61e00171a9c389ea6388024f256ad2d435da40bb370bf39e96a86d577dd36773: Status 404 returned error can't find the container with id 61e00171a9c389ea6388024f256ad2d435da40bb370bf39e96a86d577dd36773 Jun 06 22:06:46.351709 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:46.351621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" event={"ID":"e8f6cde2-e582-46f0-9d75-bbe9327855cb","Type":"ContainerStarted","Data":"61e00171a9c389ea6388024f256ad2d435da40bb370bf39e96a86d577dd36773"} Jun 06 22:06:48.360306 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:48.360218 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" event={"ID":"e8f6cde2-e582-46f0-9d75-bbe9327855cb","Type":"ContainerStarted","Data":"b1ccb822b6deeabedebdc8acde64d0dae414d1b27a6291e7bd9ac0d8baf871fc"} Jun 06 22:06:48.378974 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:48.378927 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kcnzv" podStartSLOduration=33.593420198 podStartE2EDuration="35.378912574s" totalImagesPullingTime="1.785492376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:48.377388083 +0000 UTC m=+228.125952801" watchObservedRunningTime="2026-06-06 22:06:48.378912574 +0000 UTC m=+228.127477293" Jun 06 22:06:49.881724 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:49.881688 2559 patch_prober.go:28] interesting pod/image-registry-6d8fb8b8-h4jf7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:06:49.882075 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:49.881735 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:06:55.867864 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.867831 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc"] Jun 06 22:06:55.870637 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.870621 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:55.874362 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.874341 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 06 22:06:55.874526 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.874361 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 06 22:06:55.874526 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.874362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 06 22:06:55.874526 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.874362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-m2brb\"" Jun 06 22:06:55.880645 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.880622 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc"] Jun 06 22:06:55.894628 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.894602 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hk9wz"] Jun 06 22:06:55.896687 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.896668 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:55.899292 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.899251 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 06 22:06:55.899292 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.899280 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 06 22:06:55.899292 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.899294 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l4vd7\"" Jun 06 22:06:55.899526 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.899339 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 06 22:06:55.958779 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.958748 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2xk2\" (UniqueName: \"kubernetes.io/projected/170582cb-e5e7-4259-9366-c50f727de250-kube-api-access-w2xk2\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:55.958926 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.958802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:55.958926 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.958859 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:55.958926 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:55.958880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/170582cb-e5e7-4259-9366-c50f727de250-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.059758 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-sys\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.059758 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059764 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-tls\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.059956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059781 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-root\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.059956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059862 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.059956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059904 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2xk2\" (UniqueName: \"kubernetes.io/projected/170582cb-e5e7-4259-9366-c50f727de250-kube-api-access-w2xk2\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.059956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab774aa9-ca08-4ed7-8599-53a27ef26305-metrics-client-ca\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.060079 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.059971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.060079 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060062 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qx5\" (UniqueName: \"kubernetes.io/projected/ab774aa9-ca08-4ed7-8599-53a27ef26305-kube-api-access-f4qx5\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.060186 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.060186 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/170582cb-e5e7-4259-9366-c50f727de250-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.060272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-textfile\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.060272 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:56.060195 2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 06 22:06:56.060272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060212 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-wtmp\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.060272 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:56.060247 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls podName:170582cb-e5e7-4259-9366-c50f727de250 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:56.560229647 +0000 UTC m=+236.308794344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-tzbqc" (UID: "170582cb-e5e7-4259-9366-c50f727de250") : secret "openshift-state-metrics-tls" not found Jun 06 22:06:56.060272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060268 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.060849 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.060832 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/170582cb-e5e7-4259-9366-c50f727de250-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.062511 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.062493 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.069619 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.069592 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2xk2\" (UniqueName: \"kubernetes.io/projected/170582cb-e5e7-4259-9366-c50f727de250-kube-api-access-w2xk2\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.161467 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qx5\" (UniqueName: \"kubernetes.io/projected/ab774aa9-ca08-4ed7-8599-53a27ef26305-kube-api-access-f4qx5\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161467 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161465 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-textfile\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-wtmp\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161668 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-sys\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-tls\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161782 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-textfile\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161791 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-root\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161796 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-wtmp\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.161843 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161836 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-root\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.162167 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161878 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab774aa9-ca08-4ed7-8599-53a27ef26305-sys\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.162167 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161881 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.162167 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.161945 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab774aa9-ca08-4ed7-8599-53a27ef26305-metrics-client-ca\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.162460 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.162437 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.162539 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.162457 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab774aa9-ca08-4ed7-8599-53a27ef26305-metrics-client-ca\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.163966 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.163946 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.164198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.164182 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab774aa9-ca08-4ed7-8599-53a27ef26305-node-exporter-tls\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.171623 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.171604 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qx5\" (UniqueName: \"kubernetes.io/projected/ab774aa9-ca08-4ed7-8599-53a27ef26305-kube-api-access-f4qx5\") pod \"node-exporter-hk9wz\" (UID: \"ab774aa9-ca08-4ed7-8599-53a27ef26305\") " pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.205414 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.205388 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hk9wz" Jun 06 22:06:56.215503 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:56.215475 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab774aa9_ca08_4ed7_8599_53a27ef26305.slice/crio-dddae7b2079962a98ea44ff1dcb8c5242a9e8eadef8b917f9895ee745f5a25e9 WatchSource:0}: Error finding container dddae7b2079962a98ea44ff1dcb8c5242a9e8eadef8b917f9895ee745f5a25e9: Status 404 returned error can't find the container with id dddae7b2079962a98ea44ff1dcb8c5242a9e8eadef8b917f9895ee745f5a25e9 Jun 06 22:06:56.385596 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.385562 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk9wz" event={"ID":"ab774aa9-ca08-4ed7-8599-53a27ef26305","Type":"ContainerStarted","Data":"dddae7b2079962a98ea44ff1dcb8c5242a9e8eadef8b917f9895ee745f5a25e9"} Jun 06 22:06:56.565090 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.565039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:56.565281 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:56.565193 2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 06 22:06:56.565281 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:56.565254 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls podName:170582cb-e5e7-4259-9366-c50f727de250 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:57.565239685 +0000 UTC m=+237.313804386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-tzbqc" (UID: "170582cb-e5e7-4259-9366-c50f727de250") : secret "openshift-state-metrics-tls" not found Jun 06 22:06:56.947919 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.947877 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:06:56.951999 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.951972 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:56.954654 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954625 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 06 22:06:56.954779 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954657 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 06 22:06:56.954779 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954675 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 06 22:06:56.954779 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954698 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 06 22:06:56.954779 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954710 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2z8cm\"" Jun 06 22:06:56.954779 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954664 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 06 22:06:56.955036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.954825 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 06 22:06:56.955125 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.955043 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 06 22:06:56.955125 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.955066 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 06 22:06:56.955526 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.955342 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 06 22:06:56.968160 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:56.968137 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:06:57.068893 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.068863 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069063 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.068955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-web-config\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069063 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.068981 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069063 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069003 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt7ht\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-kube-api-access-jt7ht\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069237 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069117 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069237 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069160 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069237 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069195 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069237 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069228 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069406 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069255 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069406 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069406 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069321 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-out\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069406 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069341 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.069406 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.069369 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170433 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt7ht\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-kube-api-access-jt7ht\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170458 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170518 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-out\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170620 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170644 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170706 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-web-config\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.170808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.170731 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.171200 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:57.170946 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle podName:5fb1d32d-9a75-44e2-8bae-6085958816f4 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:57.670920322 +0000 UTC m=+237.419485046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4") : configmap references non-existent config key: ca-bundle.crt Jun 06 22:06:57.171931 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:57.171526 2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 06 22:06:57.171931 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:06:57.171585 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls podName:5fb1d32d-9a75-44e2-8bae-6085958816f4 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:57.671567699 +0000 UTC m=+237.420132417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4") : secret "alertmanager-main-tls" not found Jun 06 22:06:57.172512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.172487 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.172721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.172700 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.174008 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.173981 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.174256 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.174214 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.174472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.174431 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.174740 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.174689 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-web-config\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.174953 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.174895 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.175777 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.175470 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.175777 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.175660 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-out\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.175948 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.175859 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.179560 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.179533 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt7ht\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-kube-api-access-jt7ht\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.390705 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.390676 2559 generic.go:356] "Generic (PLEG): container finished" podID="ab774aa9-ca08-4ed7-8599-53a27ef26305" containerID="a8aee34dac4997a7b518a6cf20c78de7b7206b2d296205fe8928b1a4b0493abc" exitCode=0 Jun 06 22:06:57.390891 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.390728 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk9wz" event={"ID":"ab774aa9-ca08-4ed7-8599-53a27ef26305","Type":"ContainerDied","Data":"a8aee34dac4997a7b518a6cf20c78de7b7206b2d296205fe8928b1a4b0493abc"} Jun 06 22:06:57.574402 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.574370 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:57.576736 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.576708 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/170582cb-e5e7-4259-9366-c50f727de250-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-tzbqc\" (UID: \"170582cb-e5e7-4259-9366-c50f727de250\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:57.675091 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.675057 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.675231 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.675121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.675774 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.675755 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.677470 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.677451 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.679314 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.679296 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" Jun 06 22:06:57.800862 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.800830 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc"] Jun 06 22:06:57.803804 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:57.803771 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170582cb_e5e7_4259_9366_c50f727de250.slice/crio-03855eb115cee4106bdc712a622621c2cc87856a10ad05b43095d49cbad37410 WatchSource:0}: Error finding container 03855eb115cee4106bdc712a622621c2cc87856a10ad05b43095d49cbad37410: Status 404 returned error can't find the container with id 03855eb115cee4106bdc712a622621c2cc87856a10ad05b43095d49cbad37410 Jun 06 22:06:57.864750 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.864727 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:06:57.874448 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.874421 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-866b46454b-kgf5p"] Jun 06 22:06:57.879214 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.879192 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.881789 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.881766 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 06 22:06:57.881946 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.881835 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 06 22:06:57.881946 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.881880 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 06 22:06:57.881946 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.881887 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 06 22:06:57.882123 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.881991 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 06 22:06:57.882288 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.882270 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jmxj7\"" Jun 06 22:06:57.882288 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.882280 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9ah11rknsoj1j\"" Jun 06 22:06:57.889703 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.889665 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-866b46454b-kgf5p"] Jun 06 22:06:57.977693 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977666 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20d62d13-296a-4028-a99c-92007545fa1c-metrics-client-ca\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977713 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977787 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977842 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjsxh\" (UniqueName: \"kubernetes.io/projected/20d62d13-296a-4028-a99c-92007545fa1c-kube-api-access-wjsxh\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977894 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-tls\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977918 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977941 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:57.978070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:57.977962 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-grpc-tls\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.003873 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.003852 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:06:58.006024 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:58.006002 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb1d32d_9a75_44e2_8bae_6085958816f4.slice/crio-6f77df9898bd2acc10ac335f66eedc9acc7a2ab80c6323340c6b78fdc8038ffb WatchSource:0}: Error finding container 6f77df9898bd2acc10ac335f66eedc9acc7a2ab80c6323340c6b78fdc8038ffb: Status 404 returned error can't find the container with id 6f77df9898bd2acc10ac335f66eedc9acc7a2ab80c6323340c6b78fdc8038ffb Jun 06 22:06:58.079183 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079153 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-tls\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079226 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079480 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-grpc-tls\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079566 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079523 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20d62d13-296a-4028-a99c-92007545fa1c-metrics-client-ca\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079631 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079689 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079624 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.079689 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.079667 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjsxh\" (UniqueName: \"kubernetes.io/projected/20d62d13-296a-4028-a99c-92007545fa1c-kube-api-access-wjsxh\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.080483 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.080461 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20d62d13-296a-4028-a99c-92007545fa1c-metrics-client-ca\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.082065 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.082035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-tls\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.082185 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.082035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.082446 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.082417 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.082536 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.082516 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-grpc-tls\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.082690 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.082671 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.082888 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.082869 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/20d62d13-296a-4028-a99c-92007545fa1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.088167 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.088139 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjsxh\" (UniqueName: \"kubernetes.io/projected/20d62d13-296a-4028-a99c-92007545fa1c-kube-api-access-wjsxh\") pod \"thanos-querier-866b46454b-kgf5p\" (UID: \"20d62d13-296a-4028-a99c-92007545fa1c\") " pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.200595 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.200560 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:06:58.333972 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.333937 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-866b46454b-kgf5p"] Jun 06 22:06:58.337429 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:06:58.337377 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d62d13_296a_4028_a99c_92007545fa1c.slice/crio-3ab75b03313bfe3992cf94ae827eff0757a1ef47b3ede36ca36e09d8519ac5fb WatchSource:0}: Error finding container 3ab75b03313bfe3992cf94ae827eff0757a1ef47b3ede36ca36e09d8519ac5fb: Status 404 returned error can't find the container with id 3ab75b03313bfe3992cf94ae827eff0757a1ef47b3ede36ca36e09d8519ac5fb Jun 06 22:06:58.401003 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.400961 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" event={"ID":"170582cb-e5e7-4259-9366-c50f727de250","Type":"ContainerStarted","Data":"f111a2791e2efc81f0341b61677eae04aea8633c29ac694bb38b6ea290d2cbf7"} Jun 06 22:06:58.401207 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.401072 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" event={"ID":"170582cb-e5e7-4259-9366-c50f727de250","Type":"ContainerStarted","Data":"57fee8b8abe76078424fa7eeada62c716718a070db75c6eb2f9f032346f901cf"} Jun 06 22:06:58.401207 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.401117 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" event={"ID":"170582cb-e5e7-4259-9366-c50f727de250","Type":"ContainerStarted","Data":"03855eb115cee4106bdc712a622621c2cc87856a10ad05b43095d49cbad37410"} Jun 06 22:06:58.403420 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.403393 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk9wz" event={"ID":"ab774aa9-ca08-4ed7-8599-53a27ef26305","Type":"ContainerStarted","Data":"157b326404b04a4f895616338796eaff2e6cf4b2aa26357dd33fdb34f16b29f2"} Jun 06 22:06:58.403567 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.403445 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk9wz" event={"ID":"ab774aa9-ca08-4ed7-8599-53a27ef26305","Type":"ContainerStarted","Data":"0e8f3e75adc17d3f8e327861eb51e8688003262400ddcceab046d0b97580eced"} Jun 06 22:06:58.404643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.404614 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"3ab75b03313bfe3992cf94ae827eff0757a1ef47b3ede36ca36e09d8519ac5fb"} Jun 06 22:06:58.405797 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.405771 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"6f77df9898bd2acc10ac335f66eedc9acc7a2ab80c6323340c6b78fdc8038ffb"} Jun 06 22:06:58.425742 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:58.425695 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hk9wz" podStartSLOduration=2.485127842 podStartE2EDuration="3.425682938s" totalImagesPullingTime="864.126019ms" totalInitContainerRuntime="76.429077ms" isStatefulPod=true podCreationTimestamp="2026-06-06 22:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:58.424814495 +0000 UTC m=+238.173379239" watchObservedRunningTime="2026-06-06 22:06:58.425682938 +0000 UTC m=+238.174247673" Jun 06 22:06:59.412585 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:59.412495 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" event={"ID":"170582cb-e5e7-4259-9366-c50f727de250","Type":"ContainerStarted","Data":"db05ceee0aa5c793c4f13216fd81804c87d46a77690d329d55190d1101369dcd"} Jun 06 22:06:59.414116 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:59.414065 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe" exitCode=0 Jun 06 22:06:59.414233 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:59.414218 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe"} Jun 06 22:06:59.429715 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:59.429674 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-tzbqc" podStartSLOduration=3.309018423 podStartE2EDuration="4.429656458s" totalImagesPullingTime="1.120638035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:59.427778679 +0000 UTC m=+239.176343398" watchObservedRunningTime="2026-06-06 22:06:59.429656458 +0000 UTC m=+239.178221178" Jun 06 22:06:59.880975 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:59.880939 2559 patch_prober.go:28] interesting pod/image-registry-6d8fb8b8-h4jf7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:06:59.881142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:06:59.881000 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:00.222801 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:00.222764 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:00.222985 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:00.222821 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:01.424670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.424590 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c"} Jun 06 22:07:01.425015 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.424714 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7"} Jun 06 22:07:01.425015 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.424767 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1"} Jun 06 22:07:01.425015 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.424800 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793"} Jun 06 22:07:01.425015 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.424826 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a"} Jun 06 22:07:01.426513 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.426488 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"adf3a389a740c78e53e0557070d40945f3aba353c804c2fa361aa099d58fe57f"} Jun 06 22:07:01.426594 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.426545 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"8c1aaee91540cb94c99b972f1276e820820aec36fc9490909da5cad7e98fee71"} Jun 06 22:07:01.426594 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:01.426570 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"c1a0ae2c2c0e336bb572177752164bb243a99ad9c2f065dec64dc9355806781c"} Jun 06 22:07:02.343344 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.343315 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:02.343493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.343366 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:02.432936 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.432902 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"09bf993444210e2f6f4a9bc69ba7bff1f669024a0048565f8a5a8506b1cf2411"} Jun 06 22:07:02.433371 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.432962 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"acde70994fac3269f59fa75fd3080c56c84d9cd75537403bdd5975af60740a5e"} Jun 06 22:07:02.433371 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.432987 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" event={"ID":"20d62d13-296a-4028-a99c-92007545fa1c","Type":"ContainerStarted","Data":"3dc64e88cdec4b17ecaefd01b1da40ba667778a3bf21da2fb00ed0a57967bf71"} Jun 06 22:07:02.433371 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.433032 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:07:02.436362 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.436344 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerStarted","Data":"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58"} Jun 06 22:07:02.456202 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.456159 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" podStartSLOduration=2.128674536 podStartE2EDuration="5.456146493s" totalImagesPullingTime="3.327471957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:02.454220343 +0000 UTC m=+242.202785080" watchObservedRunningTime="2026-06-06 22:07:02.456146493 +0000 UTC m=+242.204711231" Jun 06 22:07:02.477993 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:02.477955 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.261110519 podStartE2EDuration="6.477944288s" totalImagesPullingTime="3.194681037s" totalInitContainerRuntime="22.152732ms" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:02.47598899 +0000 UTC m=+242.224553735" watchObservedRunningTime="2026-06-06 22:07:02.477944288 +0000 UTC m=+242.226509006" Jun 06 22:07:04.895777 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:04.895741 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerName="registry" containerID="cri-o://a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1" gracePeriod=30 Jun 06 22:07:05.138109 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.138073 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:07:05.238932 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.238889 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvjl4\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-kube-api-access-kvjl4\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.238932 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.238925 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-installation-pull-secrets\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.238963 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-trusted-ca\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.238987 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-certificates\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.239026 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-image-registry-private-configuration\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.239041 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-bound-sa-token\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.239122 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.239150 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aa2f6a1-8045-4da3-aad4-588972d9d972-ca-trust-extracted\") pod \"4aa2f6a1-8045-4da3-aad4-588972d9d972\" (UID: \"4aa2f6a1-8045-4da3-aad4-588972d9d972\") " Jun 06 22:07:05.239554 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.239484 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:07:05.240678 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.240638 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:07:05.242000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.241958 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:07:05.242000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.241981 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:07:05.242373 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.241975 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:07:05.242373 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.242216 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-kube-api-access-kvjl4" (OuterVolumeSpecName: "kube-api-access-kvjl4") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "kube-api-access-kvjl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:07:05.242373 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.242239 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:07:05.249567 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.249540 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aa2f6a1-8045-4da3-aad4-588972d9d972-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4aa2f6a1-8045-4da3-aad4-588972d9d972" (UID: "4aa2f6a1-8045-4da3-aad4-588972d9d972"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:07:05.340343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340318 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-certificates\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340338 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-image-registry-private-configuration\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340348 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-bound-sa-token\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340358 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-registry-tls\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340367 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aa2f6a1-8045-4da3-aad4-588972d9d972-ca-trust-extracted\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340377 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvjl4\" (UniqueName: \"kubernetes.io/projected/4aa2f6a1-8045-4da3-aad4-588972d9d972-kube-api-access-kvjl4\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340385 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aa2f6a1-8045-4da3-aad4-588972d9d972-installation-pull-secrets\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.340521 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.340392 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aa2f6a1-8045-4da3-aad4-588972d9d972-trusted-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:07:05.450737 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.450711 2559 generic.go:356] "Generic (PLEG): container finished" podID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerID="a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1" exitCode=0 Jun 06 22:07:05.450846 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.450752 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" Jun 06 22:07:05.450846 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.450808 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" event={"ID":"4aa2f6a1-8045-4da3-aad4-588972d9d972","Type":"ContainerDied","Data":"a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1"} Jun 06 22:07:05.450955 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.450864 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d8fb8b8-h4jf7" event={"ID":"4aa2f6a1-8045-4da3-aad4-588972d9d972","Type":"ContainerDied","Data":"50259f0ea541e316429e3a7621c037099b5c2a38e61fe68c5361a80a99573a43"} Jun 06 22:07:05.450955 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.450895 2559 scope.go:117] "RemoveContainer" containerID="a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1" Jun 06 22:07:05.459571 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.459554 2559 scope.go:117] "RemoveContainer" containerID="a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1" Jun 06 22:07:05.459827 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:07:05.459807 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1\": container with ID starting with a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1 not found: ID does not exist" containerID="a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1" Jun 06 22:07:05.459883 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.459833 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1"} err="failed to get container status \"a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1\": rpc error: code = NotFound desc = could not find container \"a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1\": container with ID starting with a37e3e75cc6218f47cd9cf1e80f43005a623f084c410a5e7b326982213b53aa1 not found: ID does not exist" Jun 06 22:07:05.474382 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.474361 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d8fb8b8-h4jf7"] Jun 06 22:07:05.478128 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:05.478085 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d8fb8b8-h4jf7"] Jun 06 22:07:06.649158 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:06.649131 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" path="/var/lib/kubelet/pods/4aa2f6a1-8045-4da3-aad4-588972d9d972/volumes" Jun 06 22:07:08.448375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:08.448351 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-866b46454b-kgf5p" Jun 06 22:07:10.222174 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:10.222146 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:10.222504 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:10.222198 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:12.343116 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.343059 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:12.343476 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.343131 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:12.403160 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.403118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:07:12.405539 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.405512 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7635dc64-f99d-4422-99ff-419efdcefc01-metrics-certs\") pod \"network-metrics-daemon-978qx\" (UID: \"7635dc64-f99d-4422-99ff-419efdcefc01\") " pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:07:12.548421 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.548388 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d4tj5\"" Jun 06 22:07:12.557062 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.557040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-978qx" Jun 06 22:07:12.680774 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:12.680741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-978qx"] Jun 06 22:07:12.683215 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:07:12.683190 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7635dc64_f99d_4422_99ff_419efdcefc01.slice/crio-7b99c482ad9ec2fdbc27582ed8282d942ff2814222bacc8bc2a98ee89e130a70 WatchSource:0}: Error finding container 7b99c482ad9ec2fdbc27582ed8282d942ff2814222bacc8bc2a98ee89e130a70: Status 404 returned error can't find the container with id 7b99c482ad9ec2fdbc27582ed8282d942ff2814222bacc8bc2a98ee89e130a70 Jun 06 22:07:13.265883 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.265849 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-d772h"] Jun 06 22:07:13.266278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.266255 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerName="registry" Jun 06 22:07:13.266278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.266276 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerName="registry" Jun 06 22:07:13.266418 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.266352 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aa2f6a1-8045-4da3-aad4-588972d9d972" containerName="registry" Jun 06 22:07:13.268088 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.268067 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:13.270647 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.270623 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 06 22:07:13.276178 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.270674 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 06 22:07:13.276178 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.271171 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jwxvf\"" Jun 06 22:07:13.279240 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.279177 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-d772h"] Jun 06 22:07:13.310002 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.309975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xdqb\" (UniqueName: \"kubernetes.io/projected/9391c549-e6c6-4f24-ad76-7dd728cc585f-kube-api-access-9xdqb\") pod \"downloads-6657875f97-d772h\" (UID: \"9391c549-e6c6-4f24-ad76-7dd728cc585f\") " pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:13.410412 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.410376 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xdqb\" (UniqueName: \"kubernetes.io/projected/9391c549-e6c6-4f24-ad76-7dd728cc585f-kube-api-access-9xdqb\") pod \"downloads-6657875f97-d772h\" (UID: \"9391c549-e6c6-4f24-ad76-7dd728cc585f\") " pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:13.419811 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.419779 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xdqb\" (UniqueName: \"kubernetes.io/projected/9391c549-e6c6-4f24-ad76-7dd728cc585f-kube-api-access-9xdqb\") pod \"downloads-6657875f97-d772h\" (UID: \"9391c549-e6c6-4f24-ad76-7dd728cc585f\") " pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:13.481580 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.481545 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-978qx" event={"ID":"7635dc64-f99d-4422-99ff-419efdcefc01","Type":"ContainerStarted","Data":"7b99c482ad9ec2fdbc27582ed8282d942ff2814222bacc8bc2a98ee89e130a70"} Jun 06 22:07:13.582647 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.582627 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:13.720277 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:13.720248 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-d772h"] Jun 06 22:07:13.723416 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:07:13.723390 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9391c549_e6c6_4f24_ad76_7dd728cc585f.slice/crio-f8d4e9a76ad21f763778e2c08ea6232d9c93d3df88c98b026ee89ec6f549402b WatchSource:0}: Error finding container f8d4e9a76ad21f763778e2c08ea6232d9c93d3df88c98b026ee89ec6f549402b: Status 404 returned error can't find the container with id f8d4e9a76ad21f763778e2c08ea6232d9c93d3df88c98b026ee89ec6f549402b Jun 06 22:07:14.488127 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:14.487997 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-978qx" event={"ID":"7635dc64-f99d-4422-99ff-419efdcefc01","Type":"ContainerStarted","Data":"bca1261aebad8d01fff883b05e2538e7eaa3550584a5a875ba15f79b35968ef4"} Jun 06 22:07:14.488127 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:14.488067 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-978qx" event={"ID":"7635dc64-f99d-4422-99ff-419efdcefc01","Type":"ContainerStarted","Data":"e1eaab8e66841155451c6651d8e95d6b57114464ed6967207bbc6b595baef5ec"} Jun 06 22:07:14.489401 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:14.489377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-d772h" event={"ID":"9391c549-e6c6-4f24-ad76-7dd728cc585f","Type":"ContainerStarted","Data":"f8d4e9a76ad21f763778e2c08ea6232d9c93d3df88c98b026ee89ec6f549402b"} Jun 06 22:07:14.504691 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:14.504643 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-978qx" podStartSLOduration=253.696336156 podStartE2EDuration="4m14.504628741s" totalImagesPullingTime="808.292585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:14.503383227 +0000 UTC m=+254.251947968" watchObservedRunningTime="2026-06-06 22:07:14.504628741 +0000 UTC m=+254.253193460" Jun 06 22:07:20.222514 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:20.222418 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:20.222514 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:20.222472 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:20.222514 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:20.222513 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:07:20.223211 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:20.223019 2559 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"16d1b6e2e931b11e110a7f5e9703c9ab131e178daf139e8108e452fcb85bae9a"} pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" containerMessage="Container registry failed liveness probe, will be restarted" Jun 06 22:07:20.227464 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:20.227431 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:20.227584 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:20.227480 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:22.975501 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.975452 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56cd5957d9-ldhz6"] Jun 06 22:07:22.979249 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.979191 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:22.982913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.982670 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 06 22:07:22.982913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.982706 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 06 22:07:22.982913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.982714 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 06 22:07:22.982913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.982786 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jnkpl\"" Jun 06 22:07:22.982913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.982706 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 06 22:07:22.982913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.982805 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 06 22:07:22.988332 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:22.988299 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56cd5957d9-ldhz6"] Jun 06 22:07:23.088777 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.088743 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-service-ca\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.088967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.088792 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbx7c\" (UniqueName: \"kubernetes.io/projected/df5fcd13-818f-496d-9df3-adc0af25c7ed-kube-api-access-fbx7c\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.088967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.088865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-oauth-config\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.088967 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.088934 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-oauth-serving-cert\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.089155 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.088980 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-config\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.089155 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.089016 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-serving-cert\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190182 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-oauth-config\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190182 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190184 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-oauth-serving-cert\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190412 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-config\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190412 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190229 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-serving-cert\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190412 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190395 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-service-ca\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190529 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190425 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx7c\" (UniqueName: \"kubernetes.io/projected/df5fcd13-818f-496d-9df3-adc0af25c7ed-kube-api-access-fbx7c\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190996 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190937 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-oauth-serving-cert\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.190996 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.190982 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-config\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.191179 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.191018 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-service-ca\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.192949 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.192922 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-oauth-config\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.193321 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.193301 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-serving-cert\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.198603 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.198581 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbx7c\" (UniqueName: \"kubernetes.io/projected/df5fcd13-818f-496d-9df3-adc0af25c7ed-kube-api-access-fbx7c\") pod \"console-56cd5957d9-ldhz6\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.290624 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.290542 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:23.530522 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.530483 2559 generic.go:356] "Generic (PLEG): container finished" podID="48b6c9d8-995f-499c-8695-640e911922c7" containerID="c7557b1f45b2640549d75920a292dda56e97da145cfbaa89576aa89510755de6" exitCode=0 Jun 06 22:07:23.530674 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.530521 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" event={"ID":"48b6c9d8-995f-499c-8695-640e911922c7","Type":"ContainerDied","Data":"c7557b1f45b2640549d75920a292dda56e97da145cfbaa89576aa89510755de6"} Jun 06 22:07:23.530912 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:23.530892 2559 scope.go:117] "RemoveContainer" containerID="c7557b1f45b2640549d75920a292dda56e97da145cfbaa89576aa89510755de6" Jun 06 22:07:25.374663 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.374624 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d5c888df6-cnp68"] Jun 06 22:07:25.378054 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.378033 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.387611 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.387590 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 06 22:07:25.388527 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.388475 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d5c888df6-cnp68"] Jun 06 22:07:25.409448 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-oauth-config\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.409558 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409498 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-trusted-ca-bundle\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.409558 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409544 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbfmg\" (UniqueName: \"kubernetes.io/projected/a131fd0f-db26-45f6-bb2f-6d1af004667d-kube-api-access-nbfmg\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.409662 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409583 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-config\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.409662 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409623 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-serving-cert\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.409662 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-oauth-serving-cert\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.409785 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.409680 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-service-ca\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.510989 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.510956 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-config\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511192 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511014 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-serving-cert\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511264 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511210 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-oauth-serving-cert\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511321 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511267 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-service-ca\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511321 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511303 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-oauth-config\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511511 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511486 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-trusted-ca-bundle\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511585 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511564 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbfmg\" (UniqueName: \"kubernetes.io/projected/a131fd0f-db26-45f6-bb2f-6d1af004667d-kube-api-access-nbfmg\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.511875 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511824 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-config\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.512161 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.511942 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-oauth-serving-cert\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.512161 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.512076 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-service-ca\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.512392 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.512369 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-trusted-ca-bundle\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.513995 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.513975 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-oauth-config\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.514273 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.514252 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-serving-cert\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.519305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.519285 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbfmg\" (UniqueName: \"kubernetes.io/projected/a131fd0f-db26-45f6-bb2f-6d1af004667d-kube-api-access-nbfmg\") pod \"console-d5c888df6-cnp68\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:25.691241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:25.691166 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:30.091602 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.091536 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d5c888df6-cnp68"] Jun 06 22:07:30.093814 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:07:30.093627 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda131fd0f_db26_45f6_bb2f_6d1af004667d.slice/crio-22c180d3471dfcfee681fdcb25cd033c91b97665cded12e543e3852b62b12fbe WatchSource:0}: Error finding container 22c180d3471dfcfee681fdcb25cd033c91b97665cded12e543e3852b62b12fbe: Status 404 returned error can't find the container with id 22c180d3471dfcfee681fdcb25cd033c91b97665cded12e543e3852b62b12fbe Jun 06 22:07:30.110776 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.110757 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56cd5957d9-ldhz6"] Jun 06 22:07:30.114349 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:07:30.114302 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5fcd13_818f_496d_9df3_adc0af25c7ed.slice/crio-f4d13295b9e866baf2d78b840b69cc5fe90b5b1e5160517bd0a4e70b27bb720c WatchSource:0}: Error finding container f4d13295b9e866baf2d78b840b69cc5fe90b5b1e5160517bd0a4e70b27bb720c: Status 404 returned error can't find the container with id f4d13295b9e866baf2d78b840b69cc5fe90b5b1e5160517bd0a4e70b27bb720c Jun 06 22:07:30.228297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.228250 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:30.228449 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.228303 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:30.563915 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.563829 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-lmbjk" event={"ID":"48b6c9d8-995f-499c-8695-640e911922c7","Type":"ContainerStarted","Data":"d59d14d54860b40d2efcbb57f72d08a3a530ee641b000bd9e2c6a4a97838d260"} Jun 06 22:07:30.565757 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.565719 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5c888df6-cnp68" event={"ID":"a131fd0f-db26-45f6-bb2f-6d1af004667d","Type":"ContainerStarted","Data":"22c180d3471dfcfee681fdcb25cd033c91b97665cded12e543e3852b62b12fbe"} Jun 06 22:07:30.567678 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.567618 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-d772h" event={"ID":"9391c549-e6c6-4f24-ad76-7dd728cc585f","Type":"ContainerStarted","Data":"b918589929713cbe0b5c4ff7cfd65a3abd220a760e9080451d14b840f898d062"} Jun 06 22:07:30.567882 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.567837 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:30.569320 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.569277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cd5957d9-ldhz6" event={"ID":"df5fcd13-818f-496d-9df3-adc0af25c7ed","Type":"ContainerStarted","Data":"f4d13295b9e866baf2d78b840b69cc5fe90b5b1e5160517bd0a4e70b27bb720c"} Jun 06 22:07:30.582407 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.582386 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-d772h" Jun 06 22:07:30.599528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:30.599475 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-d772h" podStartSLOduration=1.252133214 podStartE2EDuration="17.599454741s" totalImagesPullingTime="16.347321527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:30.594486674 +0000 UTC m=+270.343051394" watchObservedRunningTime="2026-06-06 22:07:30.599454741 +0000 UTC m=+270.348019460" Jun 06 22:07:35.598006 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.597910 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5c888df6-cnp68" event={"ID":"a131fd0f-db26-45f6-bb2f-6d1af004667d","Type":"ContainerStarted","Data":"9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed"} Jun 06 22:07:35.599768 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.599736 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cd5957d9-ldhz6" event={"ID":"df5fcd13-818f-496d-9df3-adc0af25c7ed","Type":"ContainerStarted","Data":"e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b"} Jun 06 22:07:35.614766 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.614718 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d5c888df6-cnp68" podStartSLOduration=6.030883416 podStartE2EDuration="10.61470052s" totalImagesPullingTime="4.583817104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:35.613491036 +0000 UTC m=+275.362055773" watchObservedRunningTime="2026-06-06 22:07:35.61470052 +0000 UTC m=+275.363265240" Jun 06 22:07:35.631177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.631125 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56cd5957d9-ldhz6" podStartSLOduration=9.061364845 podStartE2EDuration="13.631091933s" totalImagesPullingTime="4.569727088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:35.62961849 +0000 UTC m=+275.378183234" watchObservedRunningTime="2026-06-06 22:07:35.631091933 +0000 UTC m=+275.379656652" Jun 06 22:07:35.691865 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.691827 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:35.692006 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.691888 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:35.693405 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.693365 2559 patch_prober.go:28] interesting pod/console-d5c888df6-cnp68 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body= Jun 06 22:07:35.693537 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:35.693429 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d5c888df6-cnp68" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" Jun 06 22:07:40.226714 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:40.226678 2559 patch_prober.go:28] interesting pod/image-registry-9b55d5d4c-fghf8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 06 22:07:40.227250 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:40.226739 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:07:43.139727 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:07:43.139684 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tmt8x" podUID="78c6ae3d-645e-4097-b9a8-68426888d9f2" Jun 06 22:07:43.290789 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:43.290757 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:43.290936 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:43.290806 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:07:43.291895 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:43.291873 2559 patch_prober.go:28] interesting pod/console-56cd5957d9-ldhz6 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" start-of-body= Jun 06 22:07:43.291994 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:43.291912 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-56cd5957d9-ldhz6" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerName="console" probeResult="failure" output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" Jun 06 22:07:43.632250 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:43.632223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tmt8x" Jun 06 22:07:45.245252 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:45.245210 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" podUID="aeb90dca-a413-4e71-a86c-49236305ea78" containerName="registry" containerID="cri-o://16d1b6e2e931b11e110a7f5e9703c9ab131e178daf139e8108e452fcb85bae9a" gracePeriod=30 Jun 06 22:07:45.692200 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:45.692085 2559 patch_prober.go:28] interesting pod/console-d5c888df6-cnp68 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body= Jun 06 22:07:45.692200 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:45.692175 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d5c888df6-cnp68" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" Jun 06 22:07:46.113356 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.113321 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:07:46.115842 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.115816 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78c6ae3d-645e-4097-b9a8-68426888d9f2-metrics-tls\") pod \"dns-default-tmt8x\" (UID: \"78c6ae3d-645e-4097-b9a8-68426888d9f2\") " pod="openshift-dns/dns-default-tmt8x" Jun 06 22:07:46.335522 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.335488 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j597l\"" Jun 06 22:07:46.343632 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.343606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tmt8x" Jun 06 22:07:46.492757 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.492725 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tmt8x"] Jun 06 22:07:46.495753 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:07:46.495723 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c6ae3d_645e_4097_b9a8_68426888d9f2.slice/crio-de882c3db7c6f659bbb717cc6a725726454853d92d249ad4d19098b2b5d17713 WatchSource:0}: Error finding container de882c3db7c6f659bbb717cc6a725726454853d92d249ad4d19098b2b5d17713: Status 404 returned error can't find the container with id de882c3db7c6f659bbb717cc6a725726454853d92d249ad4d19098b2b5d17713 Jun 06 22:07:46.646504 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.646423 2559 generic.go:356] "Generic (PLEG): container finished" podID="aeb90dca-a413-4e71-a86c-49236305ea78" containerID="16d1b6e2e931b11e110a7f5e9703c9ab131e178daf139e8108e452fcb85bae9a" exitCode=0 Jun 06 22:07:46.649837 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.649810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" event={"ID":"aeb90dca-a413-4e71-a86c-49236305ea78","Type":"ContainerDied","Data":"16d1b6e2e931b11e110a7f5e9703c9ab131e178daf139e8108e452fcb85bae9a"} Jun 06 22:07:46.649965 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:46.649854 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tmt8x" event={"ID":"78c6ae3d-645e-4097-b9a8-68426888d9f2","Type":"ContainerStarted","Data":"de882c3db7c6f659bbb717cc6a725726454853d92d249ad4d19098b2b5d17713"} Jun 06 22:07:47.653614 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:47.653577 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" event={"ID":"aeb90dca-a413-4e71-a86c-49236305ea78","Type":"ContainerStarted","Data":"5f2378e0a37e7d70e1e562353d5dbc9207a30d983160a3b39c773437d0e5a5b4"} Jun 06 22:07:47.653985 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:47.653716 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:07:48.659542 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:48.659506 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tmt8x" event={"ID":"78c6ae3d-645e-4097-b9a8-68426888d9f2","Type":"ContainerStarted","Data":"db9bda2b95afc71e52c0ea56ae2dc1ce08c521def8f66ad0e25eaebfa1565b05"} Jun 06 22:07:49.665595 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:49.665559 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tmt8x" event={"ID":"78c6ae3d-645e-4097-b9a8-68426888d9f2","Type":"ContainerStarted","Data":"698de32e3a6fe42d98288aba8c4db3033bc1c6de0904231bccef79cabf2c15c9"} Jun 06 22:07:49.666032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:49.665628 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tmt8x" Jun 06 22:07:49.681536 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:49.681488 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tmt8x" podStartSLOduration=251.722222789 podStartE2EDuration="4m13.681469568s" totalImagesPullingTime="1.959246779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:49.68142572 +0000 UTC m=+289.429990454" watchObservedRunningTime="2026-06-06 22:07:49.681469568 +0000 UTC m=+289.430034478" Jun 06 22:07:52.678160 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:52.678069 2559 generic.go:356] "Generic (PLEG): container finished" podID="e36d27da-a653-45a8-b223-a2f2118773ca" containerID="ed90234fe1d4bc5657a0d067fa16585bb57a67a9044f86e4d2b5a2b0901c6d59" exitCode=0 Jun 06 22:07:52.678160 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:52.678143 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hpmgc" event={"ID":"e36d27da-a653-45a8-b223-a2f2118773ca","Type":"ContainerDied","Data":"ed90234fe1d4bc5657a0d067fa16585bb57a67a9044f86e4d2b5a2b0901c6d59"} Jun 06 22:07:52.678539 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:52.678457 2559 scope.go:117] "RemoveContainer" containerID="ed90234fe1d4bc5657a0d067fa16585bb57a67a9044f86e4d2b5a2b0901c6d59" Jun 06 22:07:53.291116 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:53.291050 2559 patch_prober.go:28] interesting pod/console-56cd5957d9-ldhz6 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" start-of-body= Jun 06 22:07:53.291290 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:53.291126 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-56cd5957d9-ldhz6" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerName="console" probeResult="failure" output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" Jun 06 22:07:53.683385 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:53.683310 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hpmgc" event={"ID":"e36d27da-a653-45a8-b223-a2f2118773ca","Type":"ContainerStarted","Data":"64cb484f25c701bd7dbef6ffe45ce4e4c6ff8aaaabff486a95e2114094d01f0f"} Jun 06 22:07:55.271791 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.271755 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/init-config-reloader/0.log" Jun 06 22:07:55.460517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.460482 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/alertmanager/0.log" Jun 06 22:07:55.659151 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.659044 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/config-reloader/0.log" Jun 06 22:07:55.695433 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.695408 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:55.699004 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.698985 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:07:55.752114 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.752071 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56cd5957d9-ldhz6"] Jun 06 22:07:55.858870 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:55.858841 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/kube-rbac-proxy-web/0.log" Jun 06 22:07:56.058831 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:56.058806 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/kube-rbac-proxy/0.log" Jun 06 22:07:56.259131 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:56.259091 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/kube-rbac-proxy-metric/0.log" Jun 06 22:07:56.459578 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:56.459554 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5fb1d32d-9a75-44e2-8bae-6085958816f4/prom-label-proxy/0.log" Jun 06 22:07:56.661840 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:56.661808 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-kcnzv_e8f6cde2-e582-46f0-9d75-bbe9327855cb/cluster-monitoring-operator/0.log" Jun 06 22:07:59.059145 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:59.059116 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk9wz_ab774aa9-ca08-4ed7-8599-53a27ef26305/init-textfile/0.log" Jun 06 22:07:59.262836 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:59.262810 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk9wz_ab774aa9-ca08-4ed7-8599-53a27ef26305/node-exporter/0.log" Jun 06 22:07:59.461836 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:59.461806 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk9wz_ab774aa9-ca08-4ed7-8599-53a27ef26305/kube-rbac-proxy/0.log" Jun 06 22:07:59.659793 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:59.659756 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-tzbqc_170582cb-e5e7-4259-9366-c50f727de250/kube-rbac-proxy-main/0.log" Jun 06 22:07:59.671582 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:59.671555 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tmt8x" Jun 06 22:07:59.859172 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:07:59.859075 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-tzbqc_170582cb-e5e7-4259-9366-c50f727de250/kube-rbac-proxy-self/0.log" Jun 06 22:08:00.059442 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:00.059415 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-tzbqc_170582cb-e5e7-4259-9366-c50f727de250/openshift-state-metrics/0.log" Jun 06 22:08:00.589241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:00.589214 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:08:00.589372 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:00.589314 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:08:00.596863 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:00.596845 2559 kubelet.go:1648] "Image garbage collection succeeded" Jun 06 22:08:02.859945 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:02.859896 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/thanos-query/0.log" Jun 06 22:08:03.059429 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:03.059393 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy-web/0.log" Jun 06 22:08:03.259392 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:03.259354 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy/0.log" Jun 06 22:08:03.460784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:03.460760 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/prom-label-proxy/0.log" Jun 06 22:08:03.659164 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:03.659064 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy-rules/0.log" Jun 06 22:08:03.859803 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:03.859779 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy-metrics/0.log" Jun 06 22:08:04.660784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:04.660753 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56cd5957d9-ldhz6_df5fcd13-818f-496d-9df3-adc0af25c7ed/console/0.log" Jun 06 22:08:04.860160 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:04.860132 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d5c888df6-cnp68_a131fd0f-db26-45f6-bb2f-6d1af004667d/console/0.log" Jun 06 22:08:05.060966 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:05.060937 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-d772h_9391c549-e6c6-4f24-ad76-7dd728cc585f/download-server/0.log" Jun 06 22:08:06.059875 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:06.059844 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tmt8x_78c6ae3d-645e-4097-b9a8-68426888d9f2/dns/0.log" Jun 06 22:08:06.259533 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:06.259506 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tmt8x_78c6ae3d-645e-4097-b9a8-68426888d9f2/kube-rbac-proxy/0.log" Jun 06 22:08:06.459220 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:06.459193 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5v4lt_bb57a9c4-fe3e-4f7f-b452-011eee4d5173/dns-node-resolver/0.log" Jun 06 22:08:07.259327 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:07.259303 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9b55d5d4c-fghf8_aeb90dca-a413-4e71-a86c-49236305ea78/registry/0.log" Jun 06 22:08:07.459728 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:07.459699 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9b55d5d4c-fghf8_aeb90dca-a413-4e71-a86c-49236305ea78/registry/1.log" Jun 06 22:08:08.059166 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:08.059135 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xh4kn_c365b91d-ec96-452a-a635-fffdaf8f6f2c/node-ca/0.log" Jun 06 22:08:08.664177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:08.664150 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9b55d5d4c-fghf8" Jun 06 22:08:08.859037 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:08.859002 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rc8nj_78fb49bc-b80f-4ade-85be-ae550676a352/serve-healthcheck-canary/0.log" Jun 06 22:08:13.462781 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.462751 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65b8b8f8dc-xjl24"] Jun 06 22:08:13.498489 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.498458 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b8b8f8dc-xjl24"] Jun 06 22:08:13.498634 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.498564 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535244 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-service-ca\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535272 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nmd5\" (UniqueName: \"kubernetes.io/projected/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-kube-api-access-8nmd5\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-oauth-config\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535410 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-config\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-trusted-ca-bundle\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535617 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-serving-cert\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.535617 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.535571 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-oauth-serving-cert\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-serving-cert\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636646 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-oauth-serving-cert\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-service-ca\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nmd5\" (UniqueName: \"kubernetes.io/projected/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-kube-api-access-8nmd5\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-oauth-config\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636965 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636895 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-config\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.636965 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.636934 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-trusted-ca-bundle\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.637543 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.637470 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-oauth-serving-cert\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.637543 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.637504 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-service-ca\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.637543 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.637583 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-config\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.637864 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.637734 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-trusted-ca-bundle\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.639240 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.639212 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-serving-cert\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.639440 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.639418 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-oauth-config\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.643854 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.643834 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nmd5\" (UniqueName: \"kubernetes.io/projected/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-kube-api-access-8nmd5\") pod \"console-65b8b8f8dc-xjl24\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.807322 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.807257 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:13.970421 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:13.970394 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b8b8f8dc-xjl24"] Jun 06 22:08:13.973274 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:08:13.973249 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebbcbcc_a3b6_40ac_b136_12ab68d3a891.slice/crio-1894edf21ad1411073192383e1badd7f90302908f3bedec0fb96b673414074dd WatchSource:0}: Error finding container 1894edf21ad1411073192383e1badd7f90302908f3bedec0fb96b673414074dd: Status 404 returned error can't find the container with id 1894edf21ad1411073192383e1badd7f90302908f3bedec0fb96b673414074dd Jun 06 22:08:14.768414 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:14.768380 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b8b8f8dc-xjl24" event={"ID":"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891","Type":"ContainerStarted","Data":"20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096"} Jun 06 22:08:14.768770 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:14.768427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b8b8f8dc-xjl24" event={"ID":"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891","Type":"ContainerStarted","Data":"1894edf21ad1411073192383e1badd7f90302908f3bedec0fb96b673414074dd"} Jun 06 22:08:14.785889 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:14.785845 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65b8b8f8dc-xjl24" podStartSLOduration=1.78582949 podStartE2EDuration="1.78582949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:08:14.784599013 +0000 UTC m=+314.533163783" watchObservedRunningTime="2026-06-06 22:08:14.78582949 +0000 UTC m=+314.534394208" Jun 06 22:08:16.221033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.220996 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:08:16.221485 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.221361 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="alertmanager" containerID="cri-o://88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a" gracePeriod=120 Jun 06 22:08:16.221485 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.221421 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-web" containerID="cri-o://65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1" gracePeriod=120 Jun 06 22:08:16.221485 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.221442 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy" containerID="cri-o://f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7" gracePeriod=120 Jun 06 22:08:16.221639 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.221476 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="prom-label-proxy" containerID="cri-o://96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58" gracePeriod=120 Jun 06 22:08:16.221639 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.221421 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-metric" containerID="cri-o://e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c" gracePeriod=120 Jun 06 22:08:16.221639 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.221490 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="config-reloader" containerID="cri-o://69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793" gracePeriod=120 Jun 06 22:08:16.780724 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780657 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58" exitCode=0 Jun 06 22:08:16.780724 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780688 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7" exitCode=0 Jun 06 22:08:16.780724 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780695 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793" exitCode=0 Jun 06 22:08:16.780724 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780702 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a" exitCode=0 Jun 06 22:08:16.781032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780735 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58"} Jun 06 22:08:16.781032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780824 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7"} Jun 06 22:08:16.781032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780870 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793"} Jun 06 22:08:16.781032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:16.780894 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a"} Jun 06 22:08:17.500898 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.500869 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.567680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567604 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-metrics-client-ca\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567646 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-tls-assets\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567684 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-cluster-tls-config\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567702 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567737 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-out\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567757 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567784 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-volume\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567820 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-web-config\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567851 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-main-db\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567889 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt7ht\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-kube-api-access-jt7ht\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.567909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567912 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.568524 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567937 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.568524 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.567983 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5fb1d32d-9a75-44e2-8bae-6085958816f4\" (UID: \"5fb1d32d-9a75-44e2-8bae-6085958816f4\") " Jun 06 22:08:17.568524 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.568501 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:17.571192 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.571117 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:08:17.571308 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.571205 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-out" (OuterVolumeSpecName: "config-out") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:08:17.571308 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.571201 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:17.571673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.571632 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:08:17.571758 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.571706 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.572369 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.572343 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.572718 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.572692 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.572910 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.572884 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.572979 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.572926 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-kube-api-access-jt7ht" (OuterVolumeSpecName: "kube-api-access-jt7ht") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "kube-api-access-jt7ht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:08:17.573238 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.573217 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-volume" (OuterVolumeSpecName: "config-volume") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.577514 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.577447 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.584441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.584421 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-web-config" (OuterVolumeSpecName: "web-config") pod "5fb1d32d-9a75-44e2-8bae-6085958816f4" (UID: "5fb1d32d-9a75-44e2-8bae-6085958816f4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:17.669625 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669595 2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-volume\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669625 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669622 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-web-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669637 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-main-db\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669652 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt7ht\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-kube-api-access-jt7ht\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669666 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-main-tls\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669680 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669694 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669708 2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fb1d32d-9a75-44e2-8bae-6085958816f4-metrics-client-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669720 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5fb1d32d-9a75-44e2-8bae-6085958816f4-tls-assets\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669733 2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-cluster-tls-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669744 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.669753 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669752 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5fb1d32d-9a75-44e2-8bae-6085958816f4-config-out\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.670073 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.669760 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5fb1d32d-9a75-44e2-8bae-6085958816f4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:17.793277 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793246 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c" exitCode=0 Jun 06 22:08:17.793277 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793276 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerID="65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1" exitCode=0 Jun 06 22:08:17.793457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793324 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c"} Jun 06 22:08:17.793457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793340 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.793457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793433 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1"} Jun 06 22:08:17.793577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793471 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5fb1d32d-9a75-44e2-8bae-6085958816f4","Type":"ContainerDied","Data":"6f77df9898bd2acc10ac335f66eedc9acc7a2ab80c6323340c6b78fdc8038ffb"} Jun 06 22:08:17.793577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.793503 2559 scope.go:117] "RemoveContainer" containerID="96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58" Jun 06 22:08:17.801887 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.801869 2559 scope.go:117] "RemoveContainer" containerID="e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c" Jun 06 22:08:17.810769 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.810750 2559 scope.go:117] "RemoveContainer" containerID="f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7" Jun 06 22:08:17.819009 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.818958 2559 scope.go:117] "RemoveContainer" containerID="65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1" Jun 06 22:08:17.819402 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.819378 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:08:17.823314 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.823292 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:08:17.827473 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.827456 2559 scope.go:117] "RemoveContainer" containerID="69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793" Jun 06 22:08:17.834049 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.834033 2559 scope.go:117] "RemoveContainer" containerID="88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a" Jun 06 22:08:17.840932 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.840911 2559 scope.go:117] "RemoveContainer" containerID="c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe" Jun 06 22:08:17.848039 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.847959 2559 scope.go:117] "RemoveContainer" containerID="96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58" Jun 06 22:08:17.848285 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.848260 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58\": container with ID starting with 96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58 not found: ID does not exist" containerID="96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58" Jun 06 22:08:17.848372 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.848295 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58"} err="failed to get container status \"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58\": rpc error: code = NotFound desc = could not find container \"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58\": container with ID starting with 96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58 not found: ID does not exist" Jun 06 22:08:17.848372 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.848318 2559 scope.go:117] "RemoveContainer" containerID="e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c" Jun 06 22:08:17.848633 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.848615 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c\": container with ID starting with e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c not found: ID does not exist" containerID="e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c" Jun 06 22:08:17.848705 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.848640 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c"} err="failed to get container status \"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c\": rpc error: code = NotFound desc = could not find container \"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c\": container with ID starting with e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c not found: ID does not exist" Jun 06 22:08:17.848705 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.848661 2559 scope.go:117] "RemoveContainer" containerID="f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7" Jun 06 22:08:17.848956 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.848884 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7\": container with ID starting with f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7 not found: ID does not exist" containerID="f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7" Jun 06 22:08:17.848956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.848910 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7"} err="failed to get container status \"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7\": rpc error: code = NotFound desc = could not find container \"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7\": container with ID starting with f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7 not found: ID does not exist" Jun 06 22:08:17.848956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.848928 2559 scope.go:117] "RemoveContainer" containerID="65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1" Jun 06 22:08:17.849228 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.849209 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1\": container with ID starting with 65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1 not found: ID does not exist" containerID="65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1" Jun 06 22:08:17.849319 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849245 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1"} err="failed to get container status \"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1\": rpc error: code = NotFound desc = could not find container \"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1\": container with ID starting with 65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1 not found: ID does not exist" Jun 06 22:08:17.849319 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849264 2559 scope.go:117] "RemoveContainer" containerID="69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793" Jun 06 22:08:17.849537 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.849519 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793\": container with ID starting with 69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793 not found: ID does not exist" containerID="69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793" Jun 06 22:08:17.849577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849543 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793"} err="failed to get container status \"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793\": rpc error: code = NotFound desc = could not find container \"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793\": container with ID starting with 69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793 not found: ID does not exist" Jun 06 22:08:17.849577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849567 2559 scope.go:117] "RemoveContainer" containerID="88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a" Jun 06 22:08:17.849846 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.849826 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a\": container with ID starting with 88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a not found: ID does not exist" containerID="88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a" Jun 06 22:08:17.849895 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849851 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a"} err="failed to get container status \"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a\": rpc error: code = NotFound desc = could not find container \"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a\": container with ID starting with 88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a not found: ID does not exist" Jun 06 22:08:17.849895 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849867 2559 scope.go:117] "RemoveContainer" containerID="c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe" Jun 06 22:08:17.849975 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.849914 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:08:17.850090 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:17.850075 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe\": container with ID starting with c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe not found: ID does not exist" containerID="c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe" Jun 06 22:08:17.850163 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850112 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe"} err="failed to get container status \"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe\": rpc error: code = NotFound desc = could not find container \"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe\": container with ID starting with c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe not found: ID does not exist" Jun 06 22:08:17.850163 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850128 2559 scope.go:117] "RemoveContainer" containerID="96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58" Jun 06 22:08:17.850330 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850311 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="config-reloader" Jun 06 22:08:17.850370 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850332 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="config-reloader" Jun 06 22:08:17.850370 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850343 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-web" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850376 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-web" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850386 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="prom-label-proxy" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850391 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="prom-label-proxy" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850398 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="alertmanager" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850404 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="alertmanager" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850413 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850418 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy" Jun 06 22:08:17.850427 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850428 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-metric" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850433 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-metric" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850354 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58"} err="failed to get container status \"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58\": rpc error: code = NotFound desc = could not find container \"96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58\": container with ID starting with 96e83b37b9c67f3b3e1973076f5356030840fc95f7d03e3500a2427e008fcd58 not found: ID does not exist" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850447 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="init-config-reloader" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850453 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="init-config-reloader" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850503 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850511 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="config-reloader" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850518 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-web" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850524 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="kube-rbac-proxy-metric" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850530 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="prom-label-proxy" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850535 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" containerName="alertmanager" Jun 06 22:08:17.850661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850448 2559 scope.go:117] "RemoveContainer" containerID="e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c" Jun 06 22:08:17.851038 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850797 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c"} err="failed to get container status \"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c\": rpc error: code = NotFound desc = could not find container \"e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c\": container with ID starting with e687e615d2f832f073d4e25ea65a5becf4542d570706c4940d7085f461736d1c not found: ID does not exist" Jun 06 22:08:17.851038 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.850811 2559 scope.go:117] "RemoveContainer" containerID="f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7" Jun 06 22:08:17.851038 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851024 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7"} err="failed to get container status \"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7\": rpc error: code = NotFound desc = could not find container \"f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7\": container with ID starting with f5ea5a7f1923a6f2fbc9d9cea8a6f5391f156c817f46d66389292a13026316d7 not found: ID does not exist" Jun 06 22:08:17.851182 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851039 2559 scope.go:117] "RemoveContainer" containerID="65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1" Jun 06 22:08:17.851291 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851271 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1"} err="failed to get container status \"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1\": rpc error: code = NotFound desc = could not find container \"65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1\": container with ID starting with 65112fbb20b21396f73d19e6d6675ea3f49277d0f10996e2d22d02bf83a3faf1 not found: ID does not exist" Jun 06 22:08:17.851350 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851293 2559 scope.go:117] "RemoveContainer" containerID="69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793" Jun 06 22:08:17.851512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851494 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793"} err="failed to get container status \"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793\": rpc error: code = NotFound desc = could not find container \"69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793\": container with ID starting with 69bf2fd6e57673004e0f1c43a4351dca650a4155e45e57a643a8448924318793 not found: ID does not exist" Jun 06 22:08:17.851560 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851512 2559 scope.go:117] "RemoveContainer" containerID="88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a" Jun 06 22:08:17.851748 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851729 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a"} err="failed to get container status \"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a\": rpc error: code = NotFound desc = could not find container \"88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a\": container with ID starting with 88d16361593b78241f1260fab13ebac0abbfc02f2ea584a373faae965f7fe68a not found: ID does not exist" Jun 06 22:08:17.851808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851749 2559 scope.go:117] "RemoveContainer" containerID="c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe" Jun 06 22:08:17.851988 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.851971 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe"} err="failed to get container status \"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe\": rpc error: code = NotFound desc = could not find container \"c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe\": container with ID starting with c1cc7026ca9e4fd82d00c65412da75ee3b53d62b7b4fa4f1860783b3dd260ebe not found: ID does not exist" Jun 06 22:08:17.853361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.853347 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.855917 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.855897 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 06 22:08:17.856017 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.855922 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 06 22:08:17.856017 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.855900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 06 22:08:17.856252 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.856232 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 06 22:08:17.856332 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.856241 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 06 22:08:17.856332 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.856290 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2z8cm\"" Jun 06 22:08:17.856332 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.856324 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 06 22:08:17.856577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.856560 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 06 22:08:17.856638 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.856593 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 06 22:08:17.861069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.861049 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 06 22:08:17.867650 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.867628 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:08:17.972122 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/223bbe4e-2494-4e28-983e-31ae72b314b5-config-out\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972122 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972339 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972160 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/223bbe4e-2494-4e28-983e-31ae72b314b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972339 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krr6l\" (UniqueName: \"kubernetes.io/projected/223bbe4e-2494-4e28-983e-31ae72b314b5-kube-api-access-krr6l\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972339 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972227 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972339 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-web-config\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972339 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972346 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/223bbe4e-2494-4e28-983e-31ae72b314b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223bbe4e-2494-4e28-983e-31ae72b314b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972441 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972514 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972547 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:17.972586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:17.972572 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/223bbe4e-2494-4e28-983e-31ae72b314b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.073712 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073617 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223bbe4e-2494-4e28-983e-31ae72b314b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.073712 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073664 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.073712 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073696 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.073712 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073734 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/223bbe4e-2494-4e28-983e-31ae72b314b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/223bbe4e-2494-4e28-983e-31ae72b314b5-config-out\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/223bbe4e-2494-4e28-983e-31ae72b314b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073908 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krr6l\" (UniqueName: \"kubernetes.io/projected/223bbe4e-2494-4e28-983e-31ae72b314b5-kube-api-access-krr6l\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073973 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-web-config\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.073996 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.074016 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/223bbe4e-2494-4e28-983e-31ae72b314b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074496 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.074393 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/223bbe4e-2494-4e28-983e-31ae72b314b5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.074703 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.074662 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223bbe4e-2494-4e28-983e-31ae72b314b5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.075436 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.075364 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/223bbe4e-2494-4e28-983e-31ae72b314b5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.076939 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/223bbe4e-2494-4e28-983e-31ae72b314b5-config-out\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077009 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077055 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077137 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077394 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077281 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077501 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077479 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/223bbe4e-2494-4e28-983e-31ae72b314b5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077559 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077540 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-config-volume\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.077795 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.077780 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-web-config\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.078982 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.078960 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/223bbe4e-2494-4e28-983e-31ae72b314b5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.084665 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.084648 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krr6l\" (UniqueName: \"kubernetes.io/projected/223bbe4e-2494-4e28-983e-31ae72b314b5-kube-api-access-krr6l\") pod \"alertmanager-main-0\" (UID: \"223bbe4e-2494-4e28-983e-31ae72b314b5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.162735 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.162708 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 06 22:08:18.295124 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.295081 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 06 22:08:18.296435 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:08:18.296411 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223bbe4e_2494_4e28_983e_31ae72b314b5.slice/crio-838448c35f669bd7134dd8477406c8672d62228f65f73fbb00015348a5880c81 WatchSource:0}: Error finding container 838448c35f669bd7134dd8477406c8672d62228f65f73fbb00015348a5880c81: Status 404 returned error can't find the container with id 838448c35f669bd7134dd8477406c8672d62228f65f73fbb00015348a5880c81 Jun 06 22:08:18.650058 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.649987 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fb1d32d-9a75-44e2-8bae-6085958816f4" path="/var/lib/kubelet/pods/5fb1d32d-9a75-44e2-8bae-6085958816f4/volumes" Jun 06 22:08:18.799505 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.799478 2559 generic.go:356] "Generic (PLEG): container finished" podID="223bbe4e-2494-4e28-983e-31ae72b314b5" containerID="6b65f151c4ca32af9064ee878f559be3771b9de8bc8b4b583bbdee1616db93d1" exitCode=0 Jun 06 22:08:18.799643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.799520 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerDied","Data":"6b65f151c4ca32af9064ee878f559be3771b9de8bc8b4b583bbdee1616db93d1"} Jun 06 22:08:18.799643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:18.799564 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"838448c35f669bd7134dd8477406c8672d62228f65f73fbb00015348a5880c81"} Jun 06 22:08:19.806404 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.806369 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"8539c346448007c932611e680b693355e2fdfb8f5beddc5e5acf57513d3964e5"} Jun 06 22:08:19.806841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.806480 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"c5e6f76a7ac63fd37e34710dac1c8e75e71c785cef02dac8c135b826b27cb71b"} Jun 06 22:08:19.806841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.806529 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"643022f88b7f17943d4e657543efe9e5c4c218dae885d47a950ded5a6b4c05c1"} Jun 06 22:08:19.806841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.806569 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"7647ee9b6cb5b25eae9aef33c37ed0d203c463c63f71c8ab0a5d1880e5f585fc"} Jun 06 22:08:19.806841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.806609 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"306e4e442369ead90e8ffc4963b26981fe58fd2826c6c19f32be732e2c528132"} Jun 06 22:08:19.806841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.806655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"223bbe4e-2494-4e28-983e-31ae72b314b5","Type":"ContainerStarted","Data":"ab1caac8df8958341adf35bb10714d764927315169765ffd940926d65219e152"} Jun 06 22:08:19.833556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:19.833506 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.816823929 podStartE2EDuration="2.833491463s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.667534ms" isStatefulPod=false podCreationTimestamp="2026-06-06 22:08:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:08:19.83088056 +0000 UTC m=+319.579445278" watchObservedRunningTime="2026-06-06 22:08:19.833491463 +0000 UTC m=+319.582056182" Jun 06 22:08:20.771791 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:20.771753 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56cd5957d9-ldhz6" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerName="console" containerID="cri-o://e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b" gracePeriod=15 Jun 06 22:08:21.008603 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.008576 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:08:21.099244 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099174 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-oauth-config\") pod \"df5fcd13-818f-496d-9df3-adc0af25c7ed\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " Jun 06 22:08:21.099244 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099218 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-oauth-serving-cert\") pod \"df5fcd13-818f-496d-9df3-adc0af25c7ed\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " Jun 06 22:08:21.099244 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099242 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-serving-cert\") pod \"df5fcd13-818f-496d-9df3-adc0af25c7ed\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " Jun 06 22:08:21.099489 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099267 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbx7c\" (UniqueName: \"kubernetes.io/projected/df5fcd13-818f-496d-9df3-adc0af25c7ed-kube-api-access-fbx7c\") pod \"df5fcd13-818f-496d-9df3-adc0af25c7ed\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " Jun 06 22:08:21.099489 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099317 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-service-ca\") pod \"df5fcd13-818f-496d-9df3-adc0af25c7ed\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " Jun 06 22:08:21.099489 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099389 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-config\") pod \"df5fcd13-818f-496d-9df3-adc0af25c7ed\" (UID: \"df5fcd13-818f-496d-9df3-adc0af25c7ed\") " Jun 06 22:08:21.099725 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099700 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df5fcd13-818f-496d-9df3-adc0af25c7ed" (UID: "df5fcd13-818f-496d-9df3-adc0af25c7ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.099848 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099822 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "df5fcd13-818f-496d-9df3-adc0af25c7ed" (UID: "df5fcd13-818f-496d-9df3-adc0af25c7ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.100154 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.099842 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-config" (OuterVolumeSpecName: "console-config") pod "df5fcd13-818f-496d-9df3-adc0af25c7ed" (UID: "df5fcd13-818f-496d-9df3-adc0af25c7ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.101613 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.101573 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df5fcd13-818f-496d-9df3-adc0af25c7ed" (UID: "df5fcd13-818f-496d-9df3-adc0af25c7ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.101707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.101635 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df5fcd13-818f-496d-9df3-adc0af25c7ed" (UID: "df5fcd13-818f-496d-9df3-adc0af25c7ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.101707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.101661 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5fcd13-818f-496d-9df3-adc0af25c7ed-kube-api-access-fbx7c" (OuterVolumeSpecName: "kube-api-access-fbx7c") pod "df5fcd13-818f-496d-9df3-adc0af25c7ed" (UID: "df5fcd13-818f-496d-9df3-adc0af25c7ed"). InnerVolumeSpecName "kube-api-access-fbx7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:08:21.200891 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.200851 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-oauth-serving-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.200891 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.200889 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-serving-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.200891 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.200899 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbx7c\" (UniqueName: \"kubernetes.io/projected/df5fcd13-818f-496d-9df3-adc0af25c7ed-kube-api-access-fbx7c\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.201085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.200908 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-service-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.201085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.200916 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.201085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.200924 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df5fcd13-818f-496d-9df3-adc0af25c7ed-console-oauth-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.815866 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.815832 2559 generic.go:356] "Generic (PLEG): container finished" podID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerID="e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b" exitCode=0 Jun 06 22:08:21.816003 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.815879 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cd5957d9-ldhz6" Jun 06 22:08:21.816003 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.815914 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cd5957d9-ldhz6" event={"ID":"df5fcd13-818f-496d-9df3-adc0af25c7ed","Type":"ContainerDied","Data":"e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b"} Jun 06 22:08:21.816003 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.815963 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cd5957d9-ldhz6" event={"ID":"df5fcd13-818f-496d-9df3-adc0af25c7ed","Type":"ContainerDied","Data":"f4d13295b9e866baf2d78b840b69cc5fe90b5b1e5160517bd0a4e70b27bb720c"} Jun 06 22:08:21.816003 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.815984 2559 scope.go:117] "RemoveContainer" containerID="e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b" Jun 06 22:08:21.824752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.824735 2559 scope.go:117] "RemoveContainer" containerID="e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b" Jun 06 22:08:21.824987 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:21.824968 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b\": container with ID starting with e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b not found: ID does not exist" containerID="e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b" Jun 06 22:08:21.825052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.824994 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b"} err="failed to get container status \"e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b\": rpc error: code = NotFound desc = could not find container \"e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b\": container with ID starting with e54b14b7f4b8f23f3fad614e67a6bf2e8875ef9419ad6abe78bf7b9e99ab6e3b not found: ID does not exist" Jun 06 22:08:21.837681 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.837661 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56cd5957d9-ldhz6"] Jun 06 22:08:21.841154 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:21.841135 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56cd5957d9-ldhz6"] Jun 06 22:08:22.650240 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:22.650210 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" path="/var/lib/kubelet/pods/df5fcd13-818f-496d-9df3-adc0af25c7ed/volumes" Jun 06 22:08:23.807949 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:23.807902 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:23.808334 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:23.808017 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:23.812439 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:23.812419 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:23.828984 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:23.828965 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:08:23.875849 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:23.875821 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d5c888df6-cnp68"] Jun 06 22:08:48.902919 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:48.902821 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d5c888df6-cnp68" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerName="console" containerID="cri-o://9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed" gracePeriod=15 Jun 06 22:08:49.137199 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.137175 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:08:49.227958 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.227934 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-serving-cert\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.227981 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-trusted-ca-bundle\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.227997 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-service-ca\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228032 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-config\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228061 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-oauth-serving-cert\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228078 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbfmg\" (UniqueName: \"kubernetes.io/projected/a131fd0f-db26-45f6-bb2f-6d1af004667d-kube-api-access-nbfmg\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228115 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-oauth-config\") pod \"a131fd0f-db26-45f6-bb2f-6d1af004667d\" (UID: \"a131fd0f-db26-45f6-bb2f-6d1af004667d\") " Jun 06 22:08:49.228500 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228465 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:49.228613 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228502 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-config" (OuterVolumeSpecName: "console-config") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:49.228613 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228535 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:49.228742 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.228715 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-service-ca" (OuterVolumeSpecName: "service-ca") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:49.230152 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.230128 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:49.230399 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.230379 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:49.230480 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.230464 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a131fd0f-db26-45f6-bb2f-6d1af004667d-kube-api-access-nbfmg" (OuterVolumeSpecName: "kube-api-access-nbfmg") pod "a131fd0f-db26-45f6-bb2f-6d1af004667d" (UID: "a131fd0f-db26-45f6-bb2f-6d1af004667d"). InnerVolumeSpecName "kube-api-access-nbfmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:08:49.329600 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329575 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-trusted-ca-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.329600 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329597 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-service-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.329728 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329606 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.329728 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329615 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a131fd0f-db26-45f6-bb2f-6d1af004667d-oauth-serving-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.329728 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329623 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbfmg\" (UniqueName: \"kubernetes.io/projected/a131fd0f-db26-45f6-bb2f-6d1af004667d-kube-api-access-nbfmg\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.329728 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329632 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-oauth-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.329728 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.329641 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a131fd0f-db26-45f6-bb2f-6d1af004667d-console-serving-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:08:49.930940 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.930901 2559 generic.go:356] "Generic (PLEG): container finished" podID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerID="9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed" exitCode=0 Jun 06 22:08:49.931343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.930951 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5c888df6-cnp68" Jun 06 22:08:49.931343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.930973 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5c888df6-cnp68" event={"ID":"a131fd0f-db26-45f6-bb2f-6d1af004667d","Type":"ContainerDied","Data":"9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed"} Jun 06 22:08:49.931343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.931022 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5c888df6-cnp68" event={"ID":"a131fd0f-db26-45f6-bb2f-6d1af004667d","Type":"ContainerDied","Data":"22c180d3471dfcfee681fdcb25cd033c91b97665cded12e543e3852b62b12fbe"} Jun 06 22:08:49.931343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.931051 2559 scope.go:117] "RemoveContainer" containerID="9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed" Jun 06 22:08:49.939802 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.939782 2559 scope.go:117] "RemoveContainer" containerID="9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed" Jun 06 22:08:49.940074 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:08:49.940052 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed\": container with ID starting with 9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed not found: ID does not exist" containerID="9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed" Jun 06 22:08:49.940157 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.940086 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed"} err="failed to get container status \"9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed\": rpc error: code = NotFound desc = could not find container \"9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed\": container with ID starting with 9d847d438d650f0edbef184a18ff8615d3596616b4f1faaf2ecf98c78e2aa6ed not found: ID does not exist" Jun 06 22:08:49.950688 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.950665 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d5c888df6-cnp68"] Jun 06 22:08:49.954201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:49.954182 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d5c888df6-cnp68"] Jun 06 22:08:50.649381 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:08:50.649348 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" path="/var/lib/kubelet/pods/a131fd0f-db26-45f6-bb2f-6d1af004667d/volumes" Jun 06 22:09:21.771006 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.770969 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx"] Jun 06 22:09:21.771425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.771331 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerName="console" Jun 06 22:09:21.771425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.771343 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerName="console" Jun 06 22:09:21.771425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.771355 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerName="console" Jun 06 22:09:21.771425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.771362 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerName="console" Jun 06 22:09:21.771425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.771422 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5fcd13-818f-496d-9df3-adc0af25c7ed" containerName="console" Jun 06 22:09:21.771578 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.771432 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a131fd0f-db26-45f6-bb2f-6d1af004667d" containerName="console" Jun 06 22:09:21.773551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.773534 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.776172 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.776149 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7kgf8\"" Jun 06 22:09:21.776287 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.776175 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 06 22:09:21.777340 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.777321 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 06 22:09:21.786424 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.786386 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx"] Jun 06 22:09:21.872309 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.872276 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l9bm\" (UniqueName: \"kubernetes.io/projected/769b198c-100f-4b30-bc91-785a909eebdb-kube-api-access-8l9bm\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.872309 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.872316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.872511 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.872342 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.973781 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.973746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l9bm\" (UniqueName: \"kubernetes.io/projected/769b198c-100f-4b30-bc91-785a909eebdb-kube-api-access-8l9bm\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.973781 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.973786 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.973975 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.973815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.974168 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.974153 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.974259 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.974238 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:21.981799 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:21.981770 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l9bm\" (UniqueName: \"kubernetes.io/projected/769b198c-100f-4b30-bc91-785a909eebdb-kube-api-access-8l9bm\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:22.083209 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:22.083125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:22.229998 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:22.229977 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx"] Jun 06 22:09:22.232271 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:09:22.232241 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769b198c_100f_4b30_bc91_785a909eebdb.slice/crio-c59f520587ce545ff01bc645b362bce1ec471849f84ed994c6cd6cc0f3010281 WatchSource:0}: Error finding container c59f520587ce545ff01bc645b362bce1ec471849f84ed994c6cd6cc0f3010281: Status 404 returned error can't find the container with id c59f520587ce545ff01bc645b362bce1ec471849f84ed994c6cd6cc0f3010281 Jun 06 22:09:22.234534 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:22.234518 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:09:23.060356 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:23.060316 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" event={"ID":"769b198c-100f-4b30-bc91-785a909eebdb","Type":"ContainerStarted","Data":"c59f520587ce545ff01bc645b362bce1ec471849f84ed994c6cd6cc0f3010281"} Jun 06 22:09:29.092768 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:29.092728 2559 generic.go:356] "Generic (PLEG): container finished" podID="769b198c-100f-4b30-bc91-785a909eebdb" containerID="2122baa2085ac49edc2bba319937741b19850e032599ee34246bde339a2ae7a2" exitCode=0 Jun 06 22:09:29.093157 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:29.092765 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" event={"ID":"769b198c-100f-4b30-bc91-785a909eebdb","Type":"ContainerDied","Data":"2122baa2085ac49edc2bba319937741b19850e032599ee34246bde339a2ae7a2"} Jun 06 22:09:31.103701 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:31.103665 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" event={"ID":"769b198c-100f-4b30-bc91-785a909eebdb","Type":"ContainerStarted","Data":"459c14a3d0ab6958f26baa6465ec3deae42b47d029ab40b44c84987fbd906bed"} Jun 06 22:09:32.109328 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:32.109295 2559 generic.go:356] "Generic (PLEG): container finished" podID="769b198c-100f-4b30-bc91-785a909eebdb" containerID="459c14a3d0ab6958f26baa6465ec3deae42b47d029ab40b44c84987fbd906bed" exitCode=0 Jun 06 22:09:32.109878 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:32.109380 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" event={"ID":"769b198c-100f-4b30-bc91-785a909eebdb","Type":"ContainerDied","Data":"459c14a3d0ab6958f26baa6465ec3deae42b47d029ab40b44c84987fbd906bed"} Jun 06 22:09:41.149473 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:41.149437 2559 generic.go:356] "Generic (PLEG): container finished" podID="769b198c-100f-4b30-bc91-785a909eebdb" containerID="9e46197294eedab0fecdfb8eaf23f4324e5ee4a381510e8da788a3f4b16abe4f" exitCode=0 Jun 06 22:09:41.149827 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:41.149508 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" event={"ID":"769b198c-100f-4b30-bc91-785a909eebdb","Type":"ContainerDied","Data":"9e46197294eedab0fecdfb8eaf23f4324e5ee4a381510e8da788a3f4b16abe4f"} Jun 06 22:09:42.323900 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.323869 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:42.451503 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.451475 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l9bm\" (UniqueName: \"kubernetes.io/projected/769b198c-100f-4b30-bc91-785a909eebdb-kube-api-access-8l9bm\") pod \"769b198c-100f-4b30-bc91-785a909eebdb\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " Jun 06 22:09:42.451672 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.451539 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-util\") pod \"769b198c-100f-4b30-bc91-785a909eebdb\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " Jun 06 22:09:42.451672 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.451590 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-bundle\") pod \"769b198c-100f-4b30-bc91-785a909eebdb\" (UID: \"769b198c-100f-4b30-bc91-785a909eebdb\") " Jun 06 22:09:42.452293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.452215 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-bundle" (OuterVolumeSpecName: "bundle") pod "769b198c-100f-4b30-bc91-785a909eebdb" (UID: "769b198c-100f-4b30-bc91-785a909eebdb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:09:42.453770 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.453741 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769b198c-100f-4b30-bc91-785a909eebdb-kube-api-access-8l9bm" (OuterVolumeSpecName: "kube-api-access-8l9bm") pod "769b198c-100f-4b30-bc91-785a909eebdb" (UID: "769b198c-100f-4b30-bc91-785a909eebdb"). InnerVolumeSpecName "kube-api-access-8l9bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:09:42.455505 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.455484 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-util" (OuterVolumeSpecName: "util") pod "769b198c-100f-4b30-bc91-785a909eebdb" (UID: "769b198c-100f-4b30-bc91-785a909eebdb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:09:42.552441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.552417 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l9bm\" (UniqueName: \"kubernetes.io/projected/769b198c-100f-4b30-bc91-785a909eebdb-kube-api-access-8l9bm\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:09:42.552441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.552439 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:09:42.552558 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:42.552448 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/769b198c-100f-4b30-bc91-785a909eebdb-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:09:43.158806 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:43.158783 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" Jun 06 22:09:43.158806 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:43.158789 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dw9hxx" event={"ID":"769b198c-100f-4b30-bc91-785a909eebdb","Type":"ContainerDied","Data":"c59f520587ce545ff01bc645b362bce1ec471849f84ed994c6cd6cc0f3010281"} Jun 06 22:09:43.159069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:43.158828 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c59f520587ce545ff01bc645b362bce1ec471849f84ed994c6cd6cc0f3010281" Jun 06 22:09:49.495762 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.495730 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j"] Jun 06 22:09:49.496241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496195 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="extract" Jun 06 22:09:49.496241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496212 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="extract" Jun 06 22:09:49.496241 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496236 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="util" Jun 06 22:09:49.496404 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496244 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="util" Jun 06 22:09:49.496404 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496261 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="pull" Jun 06 22:09:49.496404 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496269 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="pull" Jun 06 22:09:49.496404 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.496351 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="769b198c-100f-4b30-bc91-785a909eebdb" containerName="extract" Jun 06 22:09:49.499128 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.499087 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.501550 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.501531 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:09:49.501649 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.501623 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 06 22:09:49.501709 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.501696 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-m7gwk\"" Jun 06 22:09:49.508708 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.508685 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j"] Jun 06 22:09:49.518422 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.518397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz988\" (UniqueName: \"kubernetes.io/projected/0a5c2737-986d-42c7-b5b0-607d6d2d4f0a-kube-api-access-fz988\") pod \"cert-manager-operator-controller-manager-57757f7cfc-4ht2j\" (UID: \"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.518514 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.518440 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a5c2737-986d-42c7-b5b0-607d6d2d4f0a-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-4ht2j\" (UID: \"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.618798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.618753 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a5c2737-986d-42c7-b5b0-607d6d2d4f0a-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-4ht2j\" (UID: \"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.618966 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.618834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz988\" (UniqueName: \"kubernetes.io/projected/0a5c2737-986d-42c7-b5b0-607d6d2d4f0a-kube-api-access-fz988\") pod \"cert-manager-operator-controller-manager-57757f7cfc-4ht2j\" (UID: \"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.619151 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.619126 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a5c2737-986d-42c7-b5b0-607d6d2d4f0a-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-4ht2j\" (UID: \"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.629841 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.629816 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz988\" (UniqueName: \"kubernetes.io/projected/0a5c2737-986d-42c7-b5b0-607d6d2d4f0a-kube-api-access-fz988\") pod \"cert-manager-operator-controller-manager-57757f7cfc-4ht2j\" (UID: \"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.808691 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.808595 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" Jun 06 22:09:49.941833 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:49.941810 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j"] Jun 06 22:09:49.944526 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:09:49.944499 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5c2737_986d_42c7_b5b0_607d6d2d4f0a.slice/crio-5720cfec5b7aa18eda907ad3fbe533791eb9e2f3dfc1b8a3d9ace2200370d89e WatchSource:0}: Error finding container 5720cfec5b7aa18eda907ad3fbe533791eb9e2f3dfc1b8a3d9ace2200370d89e: Status 404 returned error can't find the container with id 5720cfec5b7aa18eda907ad3fbe533791eb9e2f3dfc1b8a3d9ace2200370d89e Jun 06 22:09:50.187358 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:50.187276 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" event={"ID":"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a","Type":"ContainerStarted","Data":"5720cfec5b7aa18eda907ad3fbe533791eb9e2f3dfc1b8a3d9ace2200370d89e"} Jun 06 22:09:52.198004 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:52.197969 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" event={"ID":"0a5c2737-986d-42c7-b5b0-607d6d2d4f0a","Type":"ContainerStarted","Data":"b0aad7630cdb788533f827bcff7af2bc2be47883fd4150efffbf03bcd02e43ff"} Jun 06 22:09:52.218689 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:52.218637 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-4ht2j" podStartSLOduration=1.71447063 podStartE2EDuration="3.21862294s" totalImagesPullingTime="1.50415231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:09:52.215958085 +0000 UTC m=+411.964522816" watchObservedRunningTime="2026-06-06 22:09:52.21862294 +0000 UTC m=+411.967187659" Jun 06 22:09:53.963889 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:53.963847 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l"] Jun 06 22:09:53.967547 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:53.967525 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:53.969944 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:53.969924 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 06 22:09:53.970050 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:53.969924 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7kgf8\"" Jun 06 22:09:53.970974 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:53.970956 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 06 22:09:53.975215 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:53.975181 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l"] Jun 06 22:09:54.049610 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.049579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.049766 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.049648 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7p2g\" (UniqueName: \"kubernetes.io/projected/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-kube-api-access-k7p2g\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.049766 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.049693 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.150731 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.150707 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7p2g\" (UniqueName: \"kubernetes.io/projected/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-kube-api-access-k7p2g\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.150859 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.150746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.150859 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.150781 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.151196 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.151176 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.151238 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.151179 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.158550 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.158523 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7p2g\" (UniqueName: \"kubernetes.io/projected/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-kube-api-access-k7p2g\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.277503 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.277413 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:09:54.405463 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.405431 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l"] Jun 06 22:09:54.408431 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:09:54.408405 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ce5402_9b2f_4338_bd7e_7af0d3182ad1.slice/crio-fc9c06903b7104eb8c37aea280174e0a2e3e8d8e619281993cffddcef8e8c95e WatchSource:0}: Error finding container fc9c06903b7104eb8c37aea280174e0a2e3e8d8e619281993cffddcef8e8c95e: Status 404 returned error can't find the container with id fc9c06903b7104eb8c37aea280174e0a2e3e8d8e619281993cffddcef8e8c95e Jun 06 22:09:54.629140 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.629061 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s"] Jun 06 22:09:54.632270 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.632255 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.634781 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.634752 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mwgvs\"" Jun 06 22:09:54.634885 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.634798 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 06 22:09:54.634885 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.634812 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 06 22:09:54.641492 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.641470 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s"] Jun 06 22:09:54.653899 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.653876 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p7xf\" (UniqueName: \"kubernetes.io/projected/163aac9b-815b-4e58-a346-ca8192460a86-kube-api-access-8p7xf\") pod \"cert-manager-webhook-7dc76cbb94-kdj9s\" (UID: \"163aac9b-815b-4e58-a346-ca8192460a86\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.654018 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.653921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/163aac9b-815b-4e58-a346-ca8192460a86-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-kdj9s\" (UID: \"163aac9b-815b-4e58-a346-ca8192460a86\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.755273 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.755244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/163aac9b-815b-4e58-a346-ca8192460a86-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-kdj9s\" (UID: \"163aac9b-815b-4e58-a346-ca8192460a86\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.755403 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.755318 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p7xf\" (UniqueName: \"kubernetes.io/projected/163aac9b-815b-4e58-a346-ca8192460a86-kube-api-access-8p7xf\") pod \"cert-manager-webhook-7dc76cbb94-kdj9s\" (UID: \"163aac9b-815b-4e58-a346-ca8192460a86\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.763383 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.763355 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/163aac9b-815b-4e58-a346-ca8192460a86-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-kdj9s\" (UID: \"163aac9b-815b-4e58-a346-ca8192460a86\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.763511 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.763495 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p7xf\" (UniqueName: \"kubernetes.io/projected/163aac9b-815b-4e58-a346-ca8192460a86-kube-api-access-8p7xf\") pod \"cert-manager-webhook-7dc76cbb94-kdj9s\" (UID: \"163aac9b-815b-4e58-a346-ca8192460a86\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:54.949847 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:54.949816 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:55.075304 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:55.075280 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s"] Jun 06 22:09:55.077440 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:09:55.077411 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod163aac9b_815b_4e58_a346_ca8192460a86.slice/crio-3d39335622670e655e7bfd11d2c1effe987f9ff0ef3e066d971e98fcedd756c2 WatchSource:0}: Error finding container 3d39335622670e655e7bfd11d2c1effe987f9ff0ef3e066d971e98fcedd756c2: Status 404 returned error can't find the container with id 3d39335622670e655e7bfd11d2c1effe987f9ff0ef3e066d971e98fcedd756c2 Jun 06 22:09:55.211451 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:55.211368 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" event={"ID":"163aac9b-815b-4e58-a346-ca8192460a86","Type":"ContainerStarted","Data":"3d39335622670e655e7bfd11d2c1effe987f9ff0ef3e066d971e98fcedd756c2"} Jun 06 22:09:55.212854 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:55.212830 2559 generic.go:356] "Generic (PLEG): container finished" podID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerID="a899dce21ed737300c2fcc15d1104fb6b1abf11e906f26bd097afc7f5dd48c08" exitCode=0 Jun 06 22:09:55.212960 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:55.212909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" event={"ID":"10ce5402-9b2f-4338-bd7e-7af0d3182ad1","Type":"ContainerDied","Data":"a899dce21ed737300c2fcc15d1104fb6b1abf11e906f26bd097afc7f5dd48c08"} Jun 06 22:09:55.212960 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:55.212939 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" event={"ID":"10ce5402-9b2f-4338-bd7e-7af0d3182ad1","Type":"ContainerStarted","Data":"fc9c06903b7104eb8c37aea280174e0a2e3e8d8e619281993cffddcef8e8c95e"} Jun 06 22:09:59.232760 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:59.232728 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" event={"ID":"163aac9b-815b-4e58-a346-ca8192460a86","Type":"ContainerStarted","Data":"fddb1c70e30a4d0249d6ce99d74c474fdf1c737b0c59c260d203d910c6af7648"} Jun 06 22:09:59.233184 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:59.232868 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:09:59.234577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:59.234557 2559 generic.go:356] "Generic (PLEG): container finished" podID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerID="9388a2f8db68e9fbfaa26d57b5836afdafe3afd695484774660f04ff8782c147" exitCode=0 Jun 06 22:09:59.234659 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:59.234590 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" event={"ID":"10ce5402-9b2f-4338-bd7e-7af0d3182ad1","Type":"ContainerDied","Data":"9388a2f8db68e9fbfaa26d57b5836afdafe3afd695484774660f04ff8782c147"} Jun 06 22:09:59.248122 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:09:59.248060 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" podStartSLOduration=1.742812157 podStartE2EDuration="5.248049236s" totalImagesPullingTime="3.505237079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:09:59.247352197 +0000 UTC m=+418.995916923" watchObservedRunningTime="2026-06-06 22:09:59.248049236 +0000 UTC m=+418.996613954" Jun 06 22:10:00.241246 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:00.241214 2559 generic.go:356] "Generic (PLEG): container finished" podID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerID="134a2d55f9eec5a8e06164745cafcd95abc9663ef109c4e55174f99c37c1ec44" exitCode=0 Jun 06 22:10:00.241633 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:00.241294 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" event={"ID":"10ce5402-9b2f-4338-bd7e-7af0d3182ad1","Type":"ContainerDied","Data":"134a2d55f9eec5a8e06164745cafcd95abc9663ef109c4e55174f99c37c1ec44"} Jun 06 22:10:01.371789 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.371761 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:10:01.408726 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.408690 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-util\") pod \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " Jun 06 22:10:01.408910 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.408752 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7p2g\" (UniqueName: \"kubernetes.io/projected/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-kube-api-access-k7p2g\") pod \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " Jun 06 22:10:01.408910 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.408783 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-bundle\") pod \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\" (UID: \"10ce5402-9b2f-4338-bd7e-7af0d3182ad1\") " Jun 06 22:10:01.409264 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.409233 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-bundle" (OuterVolumeSpecName: "bundle") pod "10ce5402-9b2f-4338-bd7e-7af0d3182ad1" (UID: "10ce5402-9b2f-4338-bd7e-7af0d3182ad1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:10:01.411033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.411006 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-kube-api-access-k7p2g" (OuterVolumeSpecName: "kube-api-access-k7p2g") pod "10ce5402-9b2f-4338-bd7e-7af0d3182ad1" (UID: "10ce5402-9b2f-4338-bd7e-7af0d3182ad1"). InnerVolumeSpecName "kube-api-access-k7p2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:10:01.413612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.413590 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-util" (OuterVolumeSpecName: "util") pod "10ce5402-9b2f-4338-bd7e-7af0d3182ad1" (UID: "10ce5402-9b2f-4338-bd7e-7af0d3182ad1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:10:01.509929 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.509837 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:01.509929 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.509865 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7p2g\" (UniqueName: \"kubernetes.io/projected/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-kube-api-access-k7p2g\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:01.509929 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:01.509876 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce5402-9b2f-4338-bd7e-7af0d3182ad1-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:02.252161 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:02.252126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" event={"ID":"10ce5402-9b2f-4338-bd7e-7af0d3182ad1","Type":"ContainerDied","Data":"fc9c06903b7104eb8c37aea280174e0a2e3e8d8e619281993cffddcef8e8c95e"} Jun 06 22:10:02.252161 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:02.252148 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxdt9l" Jun 06 22:10:02.252362 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:02.252173 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc9c06903b7104eb8c37aea280174e0a2e3e8d8e619281993cffddcef8e8c95e" Jun 06 22:10:04.839934 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.839902 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-st9hf"] Jun 06 22:10:04.840426 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840407 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="util" Jun 06 22:10:04.840528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840429 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="util" Jun 06 22:10:04.840528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840445 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="pull" Jun 06 22:10:04.840528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840453 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="pull" Jun 06 22:10:04.840528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840472 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="extract" Jun 06 22:10:04.840528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840481 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="extract" Jun 06 22:10:04.840766 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.840562 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ce5402-9b2f-4338-bd7e-7af0d3182ad1" containerName="extract" Jun 06 22:10:04.844472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.844445 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:04.846629 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.846611 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-k7vbz\"" Jun 06 22:10:04.849853 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.849831 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-st9hf"] Jun 06 22:10:04.938950 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.938925 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v8gk\" (UniqueName: \"kubernetes.io/projected/5c0155cc-ac83-48e4-a8e4-10cc0e1a6503-kube-api-access-6v8gk\") pod \"cert-manager-59c77684cc-st9hf\" (UID: \"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503\") " pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:04.939053 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:04.938965 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c0155cc-ac83-48e4-a8e4-10cc0e1a6503-bound-sa-token\") pod \"cert-manager-59c77684cc-st9hf\" (UID: \"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503\") " pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:05.040065 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.040029 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v8gk\" (UniqueName: \"kubernetes.io/projected/5c0155cc-ac83-48e4-a8e4-10cc0e1a6503-kube-api-access-6v8gk\") pod \"cert-manager-59c77684cc-st9hf\" (UID: \"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503\") " pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:05.040281 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.040086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c0155cc-ac83-48e4-a8e4-10cc0e1a6503-bound-sa-token\") pod \"cert-manager-59c77684cc-st9hf\" (UID: \"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503\") " pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:05.047998 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.047967 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c0155cc-ac83-48e4-a8e4-10cc0e1a6503-bound-sa-token\") pod \"cert-manager-59c77684cc-st9hf\" (UID: \"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503\") " pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:05.048235 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.048214 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v8gk\" (UniqueName: \"kubernetes.io/projected/5c0155cc-ac83-48e4-a8e4-10cc0e1a6503-kube-api-access-6v8gk\") pod \"cert-manager-59c77684cc-st9hf\" (UID: \"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503\") " pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:05.154643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.154557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-st9hf" Jun 06 22:10:05.243526 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.243497 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-kdj9s" Jun 06 22:10:05.289436 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:05.289411 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-st9hf"] Jun 06 22:10:05.291302 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:05.291272 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c0155cc_ac83_48e4_a8e4_10cc0e1a6503.slice/crio-28e3241802db6be5eaf5638247f909a401cf14a79df500d92814f69e617941a4 WatchSource:0}: Error finding container 28e3241802db6be5eaf5638247f909a401cf14a79df500d92814f69e617941a4: Status 404 returned error can't find the container with id 28e3241802db6be5eaf5638247f909a401cf14a79df500d92814f69e617941a4 Jun 06 22:10:06.270757 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:06.270719 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-st9hf" event={"ID":"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503","Type":"ContainerStarted","Data":"0ab4b5d415908930b94b370c8b6472496cc44ad476217c6254832d74a1b973fb"} Jun 06 22:10:06.271220 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:06.270766 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-st9hf" event={"ID":"5c0155cc-ac83-48e4-a8e4-10cc0e1a6503","Type":"ContainerStarted","Data":"28e3241802db6be5eaf5638247f909a401cf14a79df500d92814f69e617941a4"} Jun 06 22:10:06.285942 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:06.285901 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-st9hf" podStartSLOduration=2.285888622 podStartE2EDuration="2.285888622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:06.284259743 +0000 UTC m=+426.032824466" watchObservedRunningTime="2026-06-06 22:10:06.285888622 +0000 UTC m=+426.034453339" Jun 06 22:10:13.426323 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.426286 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c"] Jun 06 22:10:13.431327 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.431306 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.433720 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.433701 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 06 22:10:13.433839 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.433738 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7kgf8\"" Jun 06 22:10:13.433839 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.433783 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 06 22:10:13.437187 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.437163 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c"] Jun 06 22:10:13.512064 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.512036 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2qtj\" (UniqueName: \"kubernetes.io/projected/b0acd357-ccaf-4924-8bc9-f1753cc98253-kube-api-access-m2qtj\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.512198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.512068 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.512198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.512150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.613213 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.613189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.613340 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.613273 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2qtj\" (UniqueName: \"kubernetes.io/projected/b0acd357-ccaf-4924-8bc9-f1753cc98253-kube-api-access-m2qtj\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.613340 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.613293 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.613556 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.613538 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.613622 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.613573 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.621668 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.621644 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2qtj\" (UniqueName: \"kubernetes.io/projected/b0acd357-ccaf-4924-8bc9-f1753cc98253-kube-api-access-m2qtj\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.742085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.742049 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:13.871078 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:13.871052 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c"] Jun 06 22:10:13.872761 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:13.872737 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0acd357_ccaf_4924_8bc9_f1753cc98253.slice/crio-76ef1068763ea14132d836f36651f27f9311eb3ff4982c3fb6a86419e78b445c WatchSource:0}: Error finding container 76ef1068763ea14132d836f36651f27f9311eb3ff4982c3fb6a86419e78b445c: Status 404 returned error can't find the container with id 76ef1068763ea14132d836f36651f27f9311eb3ff4982c3fb6a86419e78b445c Jun 06 22:10:14.305353 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:14.305324 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerID="9c5bfa5cd79ea3080541d59fa98934c80580bc7c09071b663b67c73ccb685dbd" exitCode=0 Jun 06 22:10:14.305520 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:14.305419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" event={"ID":"b0acd357-ccaf-4924-8bc9-f1753cc98253","Type":"ContainerDied","Data":"9c5bfa5cd79ea3080541d59fa98934c80580bc7c09071b663b67c73ccb685dbd"} Jun 06 22:10:14.305520 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:14.305470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" event={"ID":"b0acd357-ccaf-4924-8bc9-f1753cc98253","Type":"ContainerStarted","Data":"76ef1068763ea14132d836f36651f27f9311eb3ff4982c3fb6a86419e78b445c"} Jun 06 22:10:15.312280 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:15.312247 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerID="7342daf832ca2becc3315da4de865149cb2f92b848537cb4938f46e1667c2861" exitCode=0 Jun 06 22:10:15.312708 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:15.312293 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" event={"ID":"b0acd357-ccaf-4924-8bc9-f1753cc98253","Type":"ContainerDied","Data":"7342daf832ca2becc3315da4de865149cb2f92b848537cb4938f46e1667c2861"} Jun 06 22:10:16.318630 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:16.318600 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerID="1fb0c2af8ae1a8ee695224dbe97d78a0e4b9fe3063bac6c5b8a9930d3b548d34" exitCode=0 Jun 06 22:10:16.319000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:16.318648 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" event={"ID":"b0acd357-ccaf-4924-8bc9-f1753cc98253","Type":"ContainerDied","Data":"1fb0c2af8ae1a8ee695224dbe97d78a0e4b9fe3063bac6c5b8a9930d3b548d34"} Jun 06 22:10:17.453373 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.453350 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:17.544703 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.544672 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-util\") pod \"b0acd357-ccaf-4924-8bc9-f1753cc98253\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " Jun 06 22:10:17.544827 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.544733 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-bundle\") pod \"b0acd357-ccaf-4924-8bc9-f1753cc98253\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " Jun 06 22:10:17.544827 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.544804 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2qtj\" (UniqueName: \"kubernetes.io/projected/b0acd357-ccaf-4924-8bc9-f1753cc98253-kube-api-access-m2qtj\") pod \"b0acd357-ccaf-4924-8bc9-f1753cc98253\" (UID: \"b0acd357-ccaf-4924-8bc9-f1753cc98253\") " Jun 06 22:10:17.545549 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.545513 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-bundle" (OuterVolumeSpecName: "bundle") pod "b0acd357-ccaf-4924-8bc9-f1753cc98253" (UID: "b0acd357-ccaf-4924-8bc9-f1753cc98253"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:10:17.546884 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.546861 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0acd357-ccaf-4924-8bc9-f1753cc98253-kube-api-access-m2qtj" (OuterVolumeSpecName: "kube-api-access-m2qtj") pod "b0acd357-ccaf-4924-8bc9-f1753cc98253" (UID: "b0acd357-ccaf-4924-8bc9-f1753cc98253"). InnerVolumeSpecName "kube-api-access-m2qtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:10:17.550405 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.550369 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-util" (OuterVolumeSpecName: "util") pod "b0acd357-ccaf-4924-8bc9-f1753cc98253" (UID: "b0acd357-ccaf-4924-8bc9-f1753cc98253"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:10:17.646067 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.646006 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2qtj\" (UniqueName: \"kubernetes.io/projected/b0acd357-ccaf-4924-8bc9-f1753cc98253-kube-api-access-m2qtj\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:17.646067 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.646030 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:17.646067 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:17.646043 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0acd357-ccaf-4924-8bc9-f1753cc98253-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:18.335284 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:18.335249 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" event={"ID":"b0acd357-ccaf-4924-8bc9-f1753cc98253","Type":"ContainerDied","Data":"76ef1068763ea14132d836f36651f27f9311eb3ff4982c3fb6a86419e78b445c"} Jun 06 22:10:18.335445 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:18.335289 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd6f96c" Jun 06 22:10:18.335445 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:18.335295 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76ef1068763ea14132d836f36651f27f9311eb3ff4982c3fb6a86419e78b445c" Jun 06 22:10:27.313461 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313425 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j"] Jun 06 22:10:27.313906 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313843 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="extract" Jun 06 22:10:27.313906 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313858 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="extract" Jun 06 22:10:27.313906 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313868 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="pull" Jun 06 22:10:27.313906 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313873 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="pull" Jun 06 22:10:27.313906 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313888 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="util" Jun 06 22:10:27.313906 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313894 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="util" Jun 06 22:10:27.314083 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.313949 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0acd357-ccaf-4924-8bc9-f1753cc98253" containerName="extract" Jun 06 22:10:27.317018 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.317002 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.319646 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.319620 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 06 22:10:27.319646 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.319638 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 06 22:10:27.319790 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.319782 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 06 22:10:27.319862 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.319842 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 06 22:10:27.319978 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.319964 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dbwnd\"" Jun 06 22:10:27.337881 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.337857 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j"] Jun 06 22:10:27.423296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.423261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18f5a6d4-279d-49c5-a4bc-bb39413ca063-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.423440 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.423310 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18f5a6d4-279d-49c5-a4bc-bb39413ca063-manifests\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.423440 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.423333 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18f5a6d4-279d-49c5-a4bc-bb39413ca063-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.423510 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.423456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmmjh\" (UniqueName: \"kubernetes.io/projected/18f5a6d4-279d-49c5-a4bc-bb39413ca063-kube-api-access-qmmjh\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.423510 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.423489 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18f5a6d4-279d-49c5-a4bc-bb39413ca063-tmp\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.524320 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.524272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18f5a6d4-279d-49c5-a4bc-bb39413ca063-tmp\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.524493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.524328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18f5a6d4-279d-49c5-a4bc-bb39413ca063-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.524493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.524353 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18f5a6d4-279d-49c5-a4bc-bb39413ca063-manifests\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.524493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.524371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18f5a6d4-279d-49c5-a4bc-bb39413ca063-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.524656 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.524532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmmjh\" (UniqueName: \"kubernetes.io/projected/18f5a6d4-279d-49c5-a4bc-bb39413ca063-kube-api-access-qmmjh\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.524802 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.524770 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18f5a6d4-279d-49c5-a4bc-bb39413ca063-manifests\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.526598 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.526572 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18f5a6d4-279d-49c5-a4bc-bb39413ca063-tmp\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.526808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.526790 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18f5a6d4-279d-49c5-a4bc-bb39413ca063-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.526875 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.526847 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18f5a6d4-279d-49c5-a4bc-bb39413ca063-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.536123 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.536083 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmmjh\" (UniqueName: \"kubernetes.io/projected/18f5a6d4-279d-49c5-a4bc-bb39413ca063-kube-api-access-qmmjh\") pod \"opendatahub-operator-controller-manager-9c76bcc58-qs62j\" (UID: \"18f5a6d4-279d-49c5-a4bc-bb39413ca063\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.626899 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.626846 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:27.755133 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:27.755079 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j"] Jun 06 22:10:27.757827 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:27.757806 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f5a6d4_279d_49c5_a4bc_bb39413ca063.slice/crio-16e6fd6028b41e2157f2a3f5df72642ca7700d6a2e52e7f9b8ec065f4ee3bb41 WatchSource:0}: Error finding container 16e6fd6028b41e2157f2a3f5df72642ca7700d6a2e52e7f9b8ec065f4ee3bb41: Status 404 returned error can't find the container with id 16e6fd6028b41e2157f2a3f5df72642ca7700d6a2e52e7f9b8ec065f4ee3bb41 Jun 06 22:10:28.380865 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:28.380807 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" event={"ID":"18f5a6d4-279d-49c5-a4bc-bb39413ca063","Type":"ContainerStarted","Data":"16e6fd6028b41e2157f2a3f5df72642ca7700d6a2e52e7f9b8ec065f4ee3bb41"} Jun 06 22:10:31.397365 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:31.397328 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" event={"ID":"18f5a6d4-279d-49c5-a4bc-bb39413ca063","Type":"ContainerStarted","Data":"0cc35907cc35af11bd1e3dfe525bb1fc8addf9ef12afd19e720a973c08e3403b"} Jun 06 22:10:32.403747 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:32.403709 2559 generic.go:356] "Generic (PLEG): container finished" podID="18f5a6d4-279d-49c5-a4bc-bb39413ca063" containerID="0cc35907cc35af11bd1e3dfe525bb1fc8addf9ef12afd19e720a973c08e3403b" exitCode=0 Jun 06 22:10:32.404150 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:32.403783 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" event={"ID":"18f5a6d4-279d-49c5-a4bc-bb39413ca063","Type":"ContainerDied","Data":"0cc35907cc35af11bd1e3dfe525bb1fc8addf9ef12afd19e720a973c08e3403b"} Jun 06 22:10:33.410552 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:33.410519 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" event={"ID":"18f5a6d4-279d-49c5-a4bc-bb39413ca063","Type":"ContainerStarted","Data":"36e518e6a39d733eaf6ab245b078ab25d6450f815db7f6da504c2f22c086c457"} Jun 06 22:10:33.410927 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:33.410731 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:33.431370 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:33.431318 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" podStartSLOduration=2.170865475 podStartE2EDuration="6.431303273s" totalImagesPullingTime="3.275925601s" totalInitContainerRuntime="984.512197ms" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:33.429179254 +0000 UTC m=+453.177743991" watchObservedRunningTime="2026-06-06 22:10:33.431303273 +0000 UTC m=+453.179868050" Jun 06 22:10:44.417363 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:44.417328 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-qs62j" Jun 06 22:10:46.643974 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.643935 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz"] Jun 06 22:10:46.657633 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.657602 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.663060 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.663025 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 06 22:10:46.663384 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.663362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 06 22:10:46.663528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.663509 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 06 22:10:46.663602 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.663546 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xqp9g\"" Jun 06 22:10:46.663602 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.663546 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 06 22:10:46.664319 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.664299 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz"] Jun 06 22:10:46.780530 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.780494 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-4vz5j"] Jun 06 22:10:46.780815 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.780796 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75357243-0533-4e2d-82d2-33e5da952d5b-tmp\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.780870 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.780825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxqlb\" (UniqueName: \"kubernetes.io/projected/75357243-0533-4e2d-82d2-33e5da952d5b-kube-api-access-mxqlb\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.780919 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.780870 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75357243-0533-4e2d-82d2-33e5da952d5b-tls-certs\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.783822 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.783806 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:46.787069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.787043 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 06 22:10:46.787069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.787059 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-f5slq\"" Jun 06 22:10:46.798462 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.798442 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-4vz5j"] Jun 06 22:10:46.881914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.881881 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75357243-0533-4e2d-82d2-33e5da952d5b-tmp\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.881914 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.881912 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxqlb\" (UniqueName: \"kubernetes.io/projected/75357243-0533-4e2d-82d2-33e5da952d5b-kube-api-access-mxqlb\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.882142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.881935 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add74988-66e5-4395-a211-59d463702ee5-cert\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:46.882142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.881999 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75357243-0533-4e2d-82d2-33e5da952d5b-tls-certs\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.882142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.882055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hpps\" (UniqueName: \"kubernetes.io/projected/add74988-66e5-4395-a211-59d463702ee5-kube-api-access-5hpps\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:46.884188 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.884167 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75357243-0533-4e2d-82d2-33e5da952d5b-tmp\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.884455 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.884437 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75357243-0533-4e2d-82d2-33e5da952d5b-tls-certs\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.889985 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.889961 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxqlb\" (UniqueName: \"kubernetes.io/projected/75357243-0533-4e2d-82d2-33e5da952d5b-kube-api-access-mxqlb\") pod \"kube-auth-proxy-76d78c7b69-8x6nz\" (UID: \"75357243-0533-4e2d-82d2-33e5da952d5b\") " pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.952527 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.952492 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67"] Jun 06 22:10:46.956283 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.956264 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:46.958703 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.958687 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l92mj\"" Jun 06 22:10:46.958936 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.958923 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 06 22:10:46.959035 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.959010 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 06 22:10:46.959067 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.959052 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 06 22:10:46.959354 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.959339 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 06 22:10:46.959440 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.959339 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:10:46.967538 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.967519 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67"] Jun 06 22:10:46.968784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.968766 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" Jun 06 22:10:46.984750 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.984723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hpps\" (UniqueName: \"kubernetes.io/projected/add74988-66e5-4395-a211-59d463702ee5-kube-api-access-5hpps\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:46.984851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.984783 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add74988-66e5-4395-a211-59d463702ee5-cert\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:46.984909 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:10:46.984891 2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 06 22:10:46.985001 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:10:46.984988 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add74988-66e5-4395-a211-59d463702ee5-cert podName:add74988-66e5-4395-a211-59d463702ee5 nodeName:}" failed. No retries permitted until 2026-06-06 22:10:47.484967575 +0000 UTC m=+467.233532276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/add74988-66e5-4395-a211-59d463702ee5-cert") pod "odh-model-controller-6dc94f689f-4vz5j" (UID: "add74988-66e5-4395-a211-59d463702ee5") : secret "odh-model-controller-webhook-cert" not found Jun 06 22:10:46.993136 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:46.993118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hpps\" (UniqueName: \"kubernetes.io/projected/add74988-66e5-4395-a211-59d463702ee5-kube-api-access-5hpps\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:47.085621 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.085588 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdjxr\" (UniqueName: \"kubernetes.io/projected/50d6849e-44bc-4907-aece-641d47ec47b9-kube-api-access-gdjxr\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.085780 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.085642 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/50d6849e-44bc-4907-aece-641d47ec47b9-manager-config\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.085780 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.085684 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/50d6849e-44bc-4907-aece-641d47ec47b9-metrics-cert\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.085780 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.085752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50d6849e-44bc-4907-aece-641d47ec47b9-cert\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.104869 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.104843 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz"] Jun 06 22:10:47.106915 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:47.106891 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75357243_0533_4e2d_82d2_33e5da952d5b.slice/crio-48c1a2036e2d38553ae32f922dc3c4fe7a8430a6fedaebbb9fdf4a700094a6b3 WatchSource:0}: Error finding container 48c1a2036e2d38553ae32f922dc3c4fe7a8430a6fedaebbb9fdf4a700094a6b3: Status 404 returned error can't find the container with id 48c1a2036e2d38553ae32f922dc3c4fe7a8430a6fedaebbb9fdf4a700094a6b3 Jun 06 22:10:47.186993 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.186951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdjxr\" (UniqueName: \"kubernetes.io/projected/50d6849e-44bc-4907-aece-641d47ec47b9-kube-api-access-gdjxr\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.187226 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.187004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/50d6849e-44bc-4907-aece-641d47ec47b9-manager-config\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.187226 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.187149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/50d6849e-44bc-4907-aece-641d47ec47b9-metrics-cert\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.187358 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.187259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50d6849e-44bc-4907-aece-641d47ec47b9-cert\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.187702 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.187681 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/50d6849e-44bc-4907-aece-641d47ec47b9-manager-config\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.189747 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.189726 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50d6849e-44bc-4907-aece-641d47ec47b9-cert\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.189838 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.189783 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/50d6849e-44bc-4907-aece-641d47ec47b9-metrics-cert\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.203238 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.203168 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdjxr\" (UniqueName: \"kubernetes.io/projected/50d6849e-44bc-4907-aece-641d47ec47b9-kube-api-access-gdjxr\") pod \"lws-controller-manager-7774b9f8d5-m8h67\" (UID: \"50d6849e-44bc-4907-aece-641d47ec47b9\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.267003 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.266975 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:47.409809 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.409788 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67"] Jun 06 22:10:47.411888 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:47.411848 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d6849e_44bc_4907_aece_641d47ec47b9.slice/crio-541f61b1d93eb2193eb8482f7a82826c0f78ef2f765e5013b8cb1c00da23e948 WatchSource:0}: Error finding container 541f61b1d93eb2193eb8482f7a82826c0f78ef2f765e5013b8cb1c00da23e948: Status 404 returned error can't find the container with id 541f61b1d93eb2193eb8482f7a82826c0f78ef2f765e5013b8cb1c00da23e948 Jun 06 22:10:47.475582 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.475489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" event={"ID":"75357243-0533-4e2d-82d2-33e5da952d5b","Type":"ContainerStarted","Data":"48c1a2036e2d38553ae32f922dc3c4fe7a8430a6fedaebbb9fdf4a700094a6b3"} Jun 06 22:10:47.476732 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.476708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" event={"ID":"50d6849e-44bc-4907-aece-641d47ec47b9","Type":"ContainerStarted","Data":"541f61b1d93eb2193eb8482f7a82826c0f78ef2f765e5013b8cb1c00da23e948"} Jun 06 22:10:47.489187 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.489165 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add74988-66e5-4395-a211-59d463702ee5-cert\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:47.491616 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.491588 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add74988-66e5-4395-a211-59d463702ee5-cert\") pod \"odh-model-controller-6dc94f689f-4vz5j\" (UID: \"add74988-66e5-4395-a211-59d463702ee5\") " pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:47.693446 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.693402 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:47.856811 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:47.856758 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-4vz5j"] Jun 06 22:10:47.859282 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:47.859246 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd74988_66e5_4395_a211_59d463702ee5.slice/crio-9a2662a5fb87cce2f376d2cddf0ce69c5834de7c4cf67c86c263ed7f78d74232 WatchSource:0}: Error finding container 9a2662a5fb87cce2f376d2cddf0ce69c5834de7c4cf67c86c263ed7f78d74232: Status 404 returned error can't find the container with id 9a2662a5fb87cce2f376d2cddf0ce69c5834de7c4cf67c86c263ed7f78d74232 Jun 06 22:10:48.485578 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.485535 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" event={"ID":"add74988-66e5-4395-a211-59d463702ee5","Type":"ContainerStarted","Data":"9a2662a5fb87cce2f376d2cddf0ce69c5834de7c4cf67c86c263ed7f78d74232"} Jun 06 22:10:48.985208 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.985169 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g"] Jun 06 22:10:48.989482 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.989458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:48.993931 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.993393 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 06 22:10:48.993931 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.993602 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 06 22:10:48.993931 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.993784 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7kgf8\"" Jun 06 22:10:48.999349 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:48.999322 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g"] Jun 06 22:10:49.006457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.006040 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.006457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.006085 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fk9c\" (UniqueName: \"kubernetes.io/projected/73aebc2c-ed75-4104-845d-3ae639353869-kube-api-access-2fk9c\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.006457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.006194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.107535 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.107492 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.107719 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.107577 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.107719 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.107615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fk9c\" (UniqueName: \"kubernetes.io/projected/73aebc2c-ed75-4104-845d-3ae639353869-kube-api-access-2fk9c\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.108419 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.108372 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.108419 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.108400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.122988 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.122941 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fk9c\" (UniqueName: \"kubernetes.io/projected/73aebc2c-ed75-4104-845d-3ae639353869-kube-api-access-2fk9c\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:49.311658 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:49.311585 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:51.787128 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:51.787083 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g"] Jun 06 22:10:51.789375 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:10:51.789348 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73aebc2c_ed75_4104_845d_3ae639353869.slice/crio-7c9250c9ddc4e3ed99f29e4f8be4dbcdefe9307826dc8924e1d6bedffe5dfcab WatchSource:0}: Error finding container 7c9250c9ddc4e3ed99f29e4f8be4dbcdefe9307826dc8924e1d6bedffe5dfcab: Status 404 returned error can't find the container with id 7c9250c9ddc4e3ed99f29e4f8be4dbcdefe9307826dc8924e1d6bedffe5dfcab Jun 06 22:10:52.510069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.510033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" event={"ID":"add74988-66e5-4395-a211-59d463702ee5","Type":"ContainerStarted","Data":"c948b21d084fa66e9566d44d6e9f1f5acad43500d550acf3f0291d9af6e8b63a"} Jun 06 22:10:52.510257 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.510153 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:10:52.511712 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.511692 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" event={"ID":"75357243-0533-4e2d-82d2-33e5da952d5b","Type":"ContainerStarted","Data":"6f11df3840e4654ed7bd4ff81f4b9a6bbef59130a5a75d1de0f04bab14b8de04"} Jun 06 22:10:52.513120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.513083 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" event={"ID":"50d6849e-44bc-4907-aece-641d47ec47b9","Type":"ContainerStarted","Data":"19faf0a7ae42dcab591c932a69cbc29620c35c1d2dcce5f5d26bac6a6a493156"} Jun 06 22:10:52.513266 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.513154 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:10:52.514514 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.514495 2559 generic.go:356] "Generic (PLEG): container finished" podID="73aebc2c-ed75-4104-845d-3ae639353869" containerID="6443662476edc336e199e54c3146e5bea4f4fc5b0d833bf8b50bc15ee04b35a6" exitCode=0 Jun 06 22:10:52.514597 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.514521 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" event={"ID":"73aebc2c-ed75-4104-845d-3ae639353869","Type":"ContainerDied","Data":"6443662476edc336e199e54c3146e5bea4f4fc5b0d833bf8b50bc15ee04b35a6"} Jun 06 22:10:52.514597 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.514544 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" event={"ID":"73aebc2c-ed75-4104-845d-3ae639353869","Type":"ContainerStarted","Data":"7c9250c9ddc4e3ed99f29e4f8be4dbcdefe9307826dc8924e1d6bedffe5dfcab"} Jun 06 22:10:52.526574 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.526536 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" podStartSLOduration=2.742427429 podStartE2EDuration="6.526524642s" totalImagesPullingTime="3.784097213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:52.524830618 +0000 UTC m=+472.273395340" watchObservedRunningTime="2026-06-06 22:10:52.526524642 +0000 UTC m=+472.275089355" Jun 06 22:10:52.568436 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.568381 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76d78c7b69-8x6nz" podStartSLOduration=2.045721536 podStartE2EDuration="6.568365709s" totalImagesPullingTime="4.522644173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:52.566921223 +0000 UTC m=+472.315485945" watchObservedRunningTime="2026-06-06 22:10:52.568365709 +0000 UTC m=+472.316930466" Jun 06 22:10:52.584457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:52.584410 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" podStartSLOduration=2.365988936 podStartE2EDuration="6.584396063s" totalImagesPullingTime="4.218407127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:52.582227584 +0000 UTC m=+472.330792329" watchObservedRunningTime="2026-06-06 22:10:52.584396063 +0000 UTC m=+472.332960781" Jun 06 22:10:54.526139 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:54.526077 2559 generic.go:356] "Generic (PLEG): container finished" podID="73aebc2c-ed75-4104-845d-3ae639353869" containerID="501d5b3e13edc6b4daf18ed26731246775c801df0397e00db7ff4dadabc0c228" exitCode=0 Jun 06 22:10:54.526499 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:54.526161 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" event={"ID":"73aebc2c-ed75-4104-845d-3ae639353869","Type":"ContainerDied","Data":"501d5b3e13edc6b4daf18ed26731246775c801df0397e00db7ff4dadabc0c228"} Jun 06 22:10:55.533612 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:55.533582 2559 generic.go:356] "Generic (PLEG): container finished" podID="73aebc2c-ed75-4104-845d-3ae639353869" containerID="f0660af3424d4ecbfd2484872acb5bcbde207fdb8e2d4ab1ad88032b9ea3ddfd" exitCode=0 Jun 06 22:10:55.534057 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:55.533631 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" event={"ID":"73aebc2c-ed75-4104-845d-3ae639353869","Type":"ContainerDied","Data":"f0660af3424d4ecbfd2484872acb5bcbde207fdb8e2d4ab1ad88032b9ea3ddfd"} Jun 06 22:10:56.664993 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.664968 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:56.778828 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.778786 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-util\") pod \"73aebc2c-ed75-4104-845d-3ae639353869\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " Jun 06 22:10:56.779000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.778865 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-bundle\") pod \"73aebc2c-ed75-4104-845d-3ae639353869\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " Jun 06 22:10:56.779000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.778949 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fk9c\" (UniqueName: \"kubernetes.io/projected/73aebc2c-ed75-4104-845d-3ae639353869-kube-api-access-2fk9c\") pod \"73aebc2c-ed75-4104-845d-3ae639353869\" (UID: \"73aebc2c-ed75-4104-845d-3ae639353869\") " Jun 06 22:10:56.779928 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.779894 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-bundle" (OuterVolumeSpecName: "bundle") pod "73aebc2c-ed75-4104-845d-3ae639353869" (UID: "73aebc2c-ed75-4104-845d-3ae639353869"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:10:56.781440 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.781405 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73aebc2c-ed75-4104-845d-3ae639353869-kube-api-access-2fk9c" (OuterVolumeSpecName: "kube-api-access-2fk9c") pod "73aebc2c-ed75-4104-845d-3ae639353869" (UID: "73aebc2c-ed75-4104-845d-3ae639353869"). InnerVolumeSpecName "kube-api-access-2fk9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:10:56.785004 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.784982 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-util" (OuterVolumeSpecName: "util") pod "73aebc2c-ed75-4104-845d-3ae639353869" (UID: "73aebc2c-ed75-4104-845d-3ae639353869"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:10:56.879976 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.879889 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:56.879976 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.879939 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73aebc2c-ed75-4104-845d-3ae639353869-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:56.879976 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:56.879950 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fk9c\" (UniqueName: \"kubernetes.io/projected/73aebc2c-ed75-4104-845d-3ae639353869-kube-api-access-2fk9c\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:10:57.545153 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:57.545124 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" Jun 06 22:10:57.545302 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:57.545127 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06j558g" event={"ID":"73aebc2c-ed75-4104-845d-3ae639353869","Type":"ContainerDied","Data":"7c9250c9ddc4e3ed99f29e4f8be4dbcdefe9307826dc8924e1d6bedffe5dfcab"} Jun 06 22:10:57.545302 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:10:57.545270 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c9250c9ddc4e3ed99f29e4f8be4dbcdefe9307826dc8924e1d6bedffe5dfcab" Jun 06 22:11:03.520619 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.520593 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-m8h67" Jun 06 22:11:03.520997 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.520667 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-4vz5j" Jun 06 22:11:03.816846 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.816776 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94"] Jun 06 22:11:03.817137 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817124 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="extract" Jun 06 22:11:03.817176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817139 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="extract" Jun 06 22:11:03.817176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817156 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="util" Jun 06 22:11:03.817176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817161 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="util" Jun 06 22:11:03.817176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817167 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="pull" Jun 06 22:11:03.817176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817173 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="pull" Jun 06 22:11:03.817327 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.817226 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73aebc2c-ed75-4104-845d-3ae639353869" containerName="extract" Jun 06 22:11:03.821924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.821904 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:03.824874 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.824856 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7kgf8\"" Jun 06 22:11:03.825304 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.825287 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 06 22:11:03.825304 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.825292 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 06 22:11:03.839579 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.839554 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94"] Jun 06 22:11:03.941074 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.941042 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbqf5\" (UniqueName: \"kubernetes.io/projected/27273f6f-f0f6-466a-982d-e973fe27bb18-kube-api-access-pbqf5\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:03.941233 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.941077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:03.941233 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:03.941199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.041941 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.041911 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.042060 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.041966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbqf5\" (UniqueName: \"kubernetes.io/projected/27273f6f-f0f6-466a-982d-e973fe27bb18-kube-api-access-pbqf5\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.042060 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.041996 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.042391 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.042370 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.042391 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.042384 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.063589 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.063562 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbqf5\" (UniqueName: \"kubernetes.io/projected/27273f6f-f0f6-466a-982d-e973fe27bb18-kube-api-access-pbqf5\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.131741 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.131674 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:04.295741 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:11:04.295702 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27273f6f_f0f6_466a_982d_e973fe27bb18.slice/crio-f8d10eef927d8ccba9aad89f18c9590c61335fe8e1512d027076ab99df96b61c WatchSource:0}: Error finding container f8d10eef927d8ccba9aad89f18c9590c61335fe8e1512d027076ab99df96b61c: Status 404 returned error can't find the container with id f8d10eef927d8ccba9aad89f18c9590c61335fe8e1512d027076ab99df96b61c Jun 06 22:11:04.296600 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.296576 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94"] Jun 06 22:11:04.580577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.580545 2559 generic.go:356] "Generic (PLEG): container finished" podID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerID="a0dda2534ab759eb2b066f94f34050be82a499eebea7e7bd3c4571c671b596e3" exitCode=0 Jun 06 22:11:04.580928 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.580606 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" event={"ID":"27273f6f-f0f6-466a-982d-e973fe27bb18","Type":"ContainerDied","Data":"a0dda2534ab759eb2b066f94f34050be82a499eebea7e7bd3c4571c671b596e3"} Jun 06 22:11:04.580928 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:04.580642 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" event={"ID":"27273f6f-f0f6-466a-982d-e973fe27bb18","Type":"ContainerStarted","Data":"f8d10eef927d8ccba9aad89f18c9590c61335fe8e1512d027076ab99df96b61c"} Jun 06 22:11:06.592205 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:06.592171 2559 generic.go:356] "Generic (PLEG): container finished" podID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerID="7971a750796fee585df463bf54bda01946925517b48014eda362373cbf07365a" exitCode=0 Jun 06 22:11:06.592577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:06.592258 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" event={"ID":"27273f6f-f0f6-466a-982d-e973fe27bb18","Type":"ContainerDied","Data":"7971a750796fee585df463bf54bda01946925517b48014eda362373cbf07365a"} Jun 06 22:11:07.599389 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:07.599350 2559 generic.go:356] "Generic (PLEG): container finished" podID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerID="adc674c3342b512f692bba53783bcb7012a6b86710012d71b478bba11cc4eaa0" exitCode=0 Jun 06 22:11:07.599742 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:07.599416 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" event={"ID":"27273f6f-f0f6-466a-982d-e973fe27bb18","Type":"ContainerDied","Data":"adc674c3342b512f692bba53783bcb7012a6b86710012d71b478bba11cc4eaa0"} Jun 06 22:11:08.732077 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.732048 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:08.783788 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.783755 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-bundle\") pod \"27273f6f-f0f6-466a-982d-e973fe27bb18\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " Jun 06 22:11:08.783927 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.783819 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbqf5\" (UniqueName: \"kubernetes.io/projected/27273f6f-f0f6-466a-982d-e973fe27bb18-kube-api-access-pbqf5\") pod \"27273f6f-f0f6-466a-982d-e973fe27bb18\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " Jun 06 22:11:08.783927 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.783858 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-util\") pod \"27273f6f-f0f6-466a-982d-e973fe27bb18\" (UID: \"27273f6f-f0f6-466a-982d-e973fe27bb18\") " Jun 06 22:11:08.784777 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.784749 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-bundle" (OuterVolumeSpecName: "bundle") pod "27273f6f-f0f6-466a-982d-e973fe27bb18" (UID: "27273f6f-f0f6-466a-982d-e973fe27bb18"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:08.786040 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.786009 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27273f6f-f0f6-466a-982d-e973fe27bb18-kube-api-access-pbqf5" (OuterVolumeSpecName: "kube-api-access-pbqf5") pod "27273f6f-f0f6-466a-982d-e973fe27bb18" (UID: "27273f6f-f0f6-466a-982d-e973fe27bb18"). InnerVolumeSpecName "kube-api-access-pbqf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:11:08.789603 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.789578 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-util" (OuterVolumeSpecName: "util") pod "27273f6f-f0f6-466a-982d-e973fe27bb18" (UID: "27273f6f-f0f6-466a-982d-e973fe27bb18"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:08.884923 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.884829 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbqf5\" (UniqueName: \"kubernetes.io/projected/27273f6f-f0f6-466a-982d-e973fe27bb18-kube-api-access-pbqf5\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:11:08.884923 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.884866 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:11:08.884923 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:08.884879 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27273f6f-f0f6-466a-982d-e973fe27bb18-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:11:09.611181 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:09.611150 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" Jun 06 22:11:09.611330 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:09.611148 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acns94" event={"ID":"27273f6f-f0f6-466a-982d-e973fe27bb18","Type":"ContainerDied","Data":"f8d10eef927d8ccba9aad89f18c9590c61335fe8e1512d027076ab99df96b61c"} Jun 06 22:11:09.611330 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:11:09.611271 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8d10eef927d8ccba9aad89f18c9590c61335fe8e1512d027076ab99df96b61c" Jun 06 22:12:02.282050 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.281966 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q"] Jun 06 22:12:02.282493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282375 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="extract" Jun 06 22:12:02.282493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282389 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="extract" Jun 06 22:12:02.282493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282404 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="util" Jun 06 22:12:02.282493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282409 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="util" Jun 06 22:12:02.282493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282422 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="pull" Jun 06 22:12:02.282493 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282427 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="pull" Jun 06 22:12:02.282677 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.282510 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="27273f6f-f0f6-466a-982d-e973fe27bb18" containerName="extract" Jun 06 22:12:02.284733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.284717 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.287384 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.287355 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 06 22:12:02.287512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.287386 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 06 22:12:02.288373 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.288359 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-44xpd\"" Jun 06 22:12:02.293237 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.293209 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q"] Jun 06 22:12:02.321062 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.321032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.321251 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.321131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.321251 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.321160 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnhs7\" (UniqueName: \"kubernetes.io/projected/4c3a708e-c565-45bb-a322-dc7797186048-kube-api-access-nnhs7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.422463 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.422420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.422643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.422481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.422643 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.422502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnhs7\" (UniqueName: \"kubernetes.io/projected/4c3a708e-c565-45bb-a322-dc7797186048-kube-api-access-nnhs7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.422856 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.422831 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.422933 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.422877 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.430752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.430730 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnhs7\" (UniqueName: \"kubernetes.io/projected/4c3a708e-c565-45bb-a322-dc7797186048-kube-api-access-nnhs7\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.595265 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.595171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:02.686090 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.686062 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv"] Jun 06 22:12:02.688713 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.688694 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.699585 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.699561 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv"] Jun 06 22:12:02.724629 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.724602 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.724784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.724663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9glzt\" (UniqueName: \"kubernetes.io/projected/8f20015e-cd34-47d0-8330-de59913eed85-kube-api-access-9glzt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.724784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.724737 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.732284 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.732261 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q"] Jun 06 22:12:02.733380 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:02.733351 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3a708e_c565_45bb_a322_dc7797186048.slice/crio-08c0081ca282c468d0eeec770bab7d0571337b56d73620ac0cbd249eccc7b1b2 WatchSource:0}: Error finding container 08c0081ca282c468d0eeec770bab7d0571337b56d73620ac0cbd249eccc7b1b2: Status 404 returned error can't find the container with id 08c0081ca282c468d0eeec770bab7d0571337b56d73620ac0cbd249eccc7b1b2 Jun 06 22:12:02.825870 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.825835 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.826033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.825899 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.826033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.825930 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9glzt\" (UniqueName: \"kubernetes.io/projected/8f20015e-cd34-47d0-8330-de59913eed85-kube-api-access-9glzt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.826244 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.826222 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.826308 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.826281 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.835056 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.835030 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9glzt\" (UniqueName: \"kubernetes.io/projected/8f20015e-cd34-47d0-8330-de59913eed85-kube-api-access-9glzt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:02.868725 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.868662 2559 generic.go:356] "Generic (PLEG): container finished" podID="4c3a708e-c565-45bb-a322-dc7797186048" containerID="a523eb814d5677f0e135d7c1c7d3a263d8f4336e53feed130024db8c7328ac5c" exitCode=0 Jun 06 22:12:02.868814 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.868732 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" event={"ID":"4c3a708e-c565-45bb-a322-dc7797186048","Type":"ContainerDied","Data":"a523eb814d5677f0e135d7c1c7d3a263d8f4336e53feed130024db8c7328ac5c"} Jun 06 22:12:02.868814 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:02.868780 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" event={"ID":"4c3a708e-c565-45bb-a322-dc7797186048","Type":"ContainerStarted","Data":"08c0081ca282c468d0eeec770bab7d0571337b56d73620ac0cbd249eccc7b1b2"} Jun 06 22:12:03.000347 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.000315 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:03.083907 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.083867 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65"] Jun 06 22:12:03.087401 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.087381 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.095865 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.095840 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65"] Jun 06 22:12:03.129128 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.129032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.129128 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.129073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.129297 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.129156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwn9h\" (UniqueName: \"kubernetes.io/projected/d14f54af-3e4e-4f19-bce8-fea912d2f999-kube-api-access-lwn9h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.135576 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.135554 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv"] Jun 06 22:12:03.136957 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:03.136933 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f20015e_cd34_47d0_8330_de59913eed85.slice/crio-5e89be458a4064b5d37c362992cd25c13a63da8394e93937e0e32de2aa3e654d WatchSource:0}: Error finding container 5e89be458a4064b5d37c362992cd25c13a63da8394e93937e0e32de2aa3e654d: Status 404 returned error can't find the container with id 5e89be458a4064b5d37c362992cd25c13a63da8394e93937e0e32de2aa3e654d Jun 06 22:12:03.230264 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.230231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.230438 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.230271 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.230438 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.230298 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwn9h\" (UniqueName: \"kubernetes.io/projected/d14f54af-3e4e-4f19-bce8-fea912d2f999-kube-api-access-lwn9h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.230630 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.230609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.230700 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.230676 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.241252 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.241226 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwn9h\" (UniqueName: \"kubernetes.io/projected/d14f54af-3e4e-4f19-bce8-fea912d2f999-kube-api-access-lwn9h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.398672 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.398586 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:03.479972 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.479944 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr"] Jun 06 22:12:03.482430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.482412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.492873 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.492843 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr"] Jun 06 22:12:03.533085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.533055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.533252 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.533133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.533252 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.533217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rk7\" (UniqueName: \"kubernetes.io/projected/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-kube-api-access-g4rk7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.543842 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.543817 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65"] Jun 06 22:12:03.580015 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:03.579986 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14f54af_3e4e_4f19_bce8_fea912d2f999.slice/crio-e74662da0a037352baed14246e24a8a6fc35108baa44291041711e5ba99de67e WatchSource:0}: Error finding container e74662da0a037352baed14246e24a8a6fc35108baa44291041711e5ba99de67e: Status 404 returned error can't find the container with id e74662da0a037352baed14246e24a8a6fc35108baa44291041711e5ba99de67e Jun 06 22:12:03.634022 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.634000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.634137 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.634060 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rk7\" (UniqueName: \"kubernetes.io/projected/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-kube-api-access-g4rk7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.634137 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.634115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.634358 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.634337 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.634425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.634377 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.643218 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.643187 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rk7\" (UniqueName: \"kubernetes.io/projected/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-kube-api-access-g4rk7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.795829 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.795801 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:03.877082 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.876719 2559 generic.go:356] "Generic (PLEG): container finished" podID="4c3a708e-c565-45bb-a322-dc7797186048" containerID="e158fa8b0b30a84052e66ea0b774a72d22c8765f616e2e818f274e3c3d62cc2b" exitCode=0 Jun 06 22:12:03.877082 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.876765 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" event={"ID":"4c3a708e-c565-45bb-a322-dc7797186048","Type":"ContainerDied","Data":"e158fa8b0b30a84052e66ea0b774a72d22c8765f616e2e818f274e3c3d62cc2b"} Jun 06 22:12:03.879365 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.879241 2559 generic.go:356] "Generic (PLEG): container finished" podID="8f20015e-cd34-47d0-8330-de59913eed85" containerID="1a7c31719f901be2b88e22fb5e34bd71d32ddd550e48fb78c8cef98ca2ef6dbd" exitCode=0 Jun 06 22:12:03.879365 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.879335 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" event={"ID":"8f20015e-cd34-47d0-8330-de59913eed85","Type":"ContainerDied","Data":"1a7c31719f901be2b88e22fb5e34bd71d32ddd550e48fb78c8cef98ca2ef6dbd"} Jun 06 22:12:03.879528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.879381 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" event={"ID":"8f20015e-cd34-47d0-8330-de59913eed85","Type":"ContainerStarted","Data":"5e89be458a4064b5d37c362992cd25c13a63da8394e93937e0e32de2aa3e654d"} Jun 06 22:12:03.881949 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.881918 2559 generic.go:356] "Generic (PLEG): container finished" podID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerID="e553ea331a649856b3b2a38d6fc6eeb8a246e4ce02124ee3a03851ee491f0304" exitCode=0 Jun 06 22:12:03.882042 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.882005 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" event={"ID":"d14f54af-3e4e-4f19-bce8-fea912d2f999","Type":"ContainerDied","Data":"e553ea331a649856b3b2a38d6fc6eeb8a246e4ce02124ee3a03851ee491f0304"} Jun 06 22:12:03.882042 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.882038 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" event={"ID":"d14f54af-3e4e-4f19-bce8-fea912d2f999","Type":"ContainerStarted","Data":"e74662da0a037352baed14246e24a8a6fc35108baa44291041711e5ba99de67e"} Jun 06 22:12:03.928689 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:03.928665 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr"] Jun 06 22:12:03.929971 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:03.929947 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc51113_5723_4d8c_80f4_df8b6b7bb76f.slice/crio-f697f11638a6f62236b4cff9885e69e3d599567c3976f371d04615b3e3254989 WatchSource:0}: Error finding container f697f11638a6f62236b4cff9885e69e3d599567c3976f371d04615b3e3254989: Status 404 returned error can't find the container with id f697f11638a6f62236b4cff9885e69e3d599567c3976f371d04615b3e3254989 Jun 06 22:12:04.889177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.889148 2559 generic.go:356] "Generic (PLEG): container finished" podID="8f20015e-cd34-47d0-8330-de59913eed85" containerID="b462bd0fa5814d446aa07888c75816f8747056c35e8452360c5e2047cf9001f5" exitCode=0 Jun 06 22:12:04.889613 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.889181 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" event={"ID":"8f20015e-cd34-47d0-8330-de59913eed85","Type":"ContainerDied","Data":"b462bd0fa5814d446aa07888c75816f8747056c35e8452360c5e2047cf9001f5"} Jun 06 22:12:04.891057 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.891037 2559 generic.go:356] "Generic (PLEG): container finished" podID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerID="1f360b6956f40f4a8ab426151eb345c952925a324dfa57803dd46746087c0636" exitCode=0 Jun 06 22:12:04.891182 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.891081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" event={"ID":"d14f54af-3e4e-4f19-bce8-fea912d2f999","Type":"ContainerDied","Data":"1f360b6956f40f4a8ab426151eb345c952925a324dfa57803dd46746087c0636"} Jun 06 22:12:04.893395 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.893373 2559 generic.go:356] "Generic (PLEG): container finished" podID="4c3a708e-c565-45bb-a322-dc7797186048" containerID="02771e472c0c4c14c062e49c1279f588b9fd12271534113aa7a22a26ddf9a0d3" exitCode=0 Jun 06 22:12:04.893478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.893428 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" event={"ID":"4c3a708e-c565-45bb-a322-dc7797186048","Type":"ContainerDied","Data":"02771e472c0c4c14c062e49c1279f588b9fd12271534113aa7a22a26ddf9a0d3"} Jun 06 22:12:04.894837 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.894810 2559 generic.go:356] "Generic (PLEG): container finished" podID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerID="3208e3edd8dda12451b3a52287bfb4ac59c0219c0c9bd8238f5044091b23d640" exitCode=0 Jun 06 22:12:04.894924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.894861 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" event={"ID":"dfc51113-5723-4d8c-80f4-df8b6b7bb76f","Type":"ContainerDied","Data":"3208e3edd8dda12451b3a52287bfb4ac59c0219c0c9bd8238f5044091b23d640"} Jun 06 22:12:04.894924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:04.894905 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" event={"ID":"dfc51113-5723-4d8c-80f4-df8b6b7bb76f","Type":"ContainerStarted","Data":"f697f11638a6f62236b4cff9885e69e3d599567c3976f371d04615b3e3254989"} Jun 06 22:12:05.907925 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:05.907886 2559 generic.go:356] "Generic (PLEG): container finished" podID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerID="75c643c0731983879dd383a8a62a20ccb49d7d2b375fa80d860d2312a0150587" exitCode=0 Jun 06 22:12:05.908446 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:05.907966 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" event={"ID":"dfc51113-5723-4d8c-80f4-df8b6b7bb76f","Type":"ContainerDied","Data":"75c643c0731983879dd383a8a62a20ccb49d7d2b375fa80d860d2312a0150587"} Jun 06 22:12:05.910455 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:05.910432 2559 generic.go:356] "Generic (PLEG): container finished" podID="8f20015e-cd34-47d0-8330-de59913eed85" containerID="79a74dc18a30603a40c01b893c46c6021beb64e21ee829fecea94c2549e57c7a" exitCode=0 Jun 06 22:12:05.910553 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:05.910516 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" event={"ID":"8f20015e-cd34-47d0-8330-de59913eed85","Type":"ContainerDied","Data":"79a74dc18a30603a40c01b893c46c6021beb64e21ee829fecea94c2549e57c7a"} Jun 06 22:12:05.912595 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:05.912574 2559 generic.go:356] "Generic (PLEG): container finished" podID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerID="8d9c4b24bc914875af64d6f820b3673694f2fb115ba245f63c470335f1160d8b" exitCode=0 Jun 06 22:12:05.912678 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:05.912652 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" event={"ID":"d14f54af-3e4e-4f19-bce8-fea912d2f999","Type":"ContainerDied","Data":"8d9c4b24bc914875af64d6f820b3673694f2fb115ba245f63c470335f1160d8b"} Jun 06 22:12:06.065935 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.065905 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:06.154543 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.154513 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-bundle\") pod \"4c3a708e-c565-45bb-a322-dc7797186048\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " Jun 06 22:12:06.154543 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.154549 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-util\") pod \"4c3a708e-c565-45bb-a322-dc7797186048\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " Jun 06 22:12:06.154793 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.154607 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnhs7\" (UniqueName: \"kubernetes.io/projected/4c3a708e-c565-45bb-a322-dc7797186048-kube-api-access-nnhs7\") pod \"4c3a708e-c565-45bb-a322-dc7797186048\" (UID: \"4c3a708e-c565-45bb-a322-dc7797186048\") " Jun 06 22:12:06.155056 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.155030 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-bundle" (OuterVolumeSpecName: "bundle") pod "4c3a708e-c565-45bb-a322-dc7797186048" (UID: "4c3a708e-c565-45bb-a322-dc7797186048"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:06.156924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.156899 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3a708e-c565-45bb-a322-dc7797186048-kube-api-access-nnhs7" (OuterVolumeSpecName: "kube-api-access-nnhs7") pod "4c3a708e-c565-45bb-a322-dc7797186048" (UID: "4c3a708e-c565-45bb-a322-dc7797186048"). InnerVolumeSpecName "kube-api-access-nnhs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:12:06.160592 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.160521 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-util" (OuterVolumeSpecName: "util") pod "4c3a708e-c565-45bb-a322-dc7797186048" (UID: "4c3a708e-c565-45bb-a322-dc7797186048"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:06.255955 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.255916 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:06.255955 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.255944 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c3a708e-c565-45bb-a322-dc7797186048-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:06.255955 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.255953 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnhs7\" (UniqueName: \"kubernetes.io/projected/4c3a708e-c565-45bb-a322-dc7797186048-kube-api-access-nnhs7\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:06.920399 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.920357 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" event={"ID":"4c3a708e-c565-45bb-a322-dc7797186048","Type":"ContainerDied","Data":"08c0081ca282c468d0eeec770bab7d0571337b56d73620ac0cbd249eccc7b1b2"} Jun 06 22:12:06.920819 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.920415 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08c0081ca282c468d0eeec770bab7d0571337b56d73620ac0cbd249eccc7b1b2" Jun 06 22:12:06.920819 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.920364 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q" Jun 06 22:12:06.922685 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.922661 2559 generic.go:356] "Generic (PLEG): container finished" podID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerID="1c5c3df4ef6ab594949be2abdff70ba4a2cd9518ba9712c744edd02f3f1c5f4b" exitCode=0 Jun 06 22:12:06.922784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:06.922749 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" event={"ID":"dfc51113-5723-4d8c-80f4-df8b6b7bb76f","Type":"ContainerDied","Data":"1c5c3df4ef6ab594949be2abdff70ba4a2cd9518ba9712c744edd02f3f1c5f4b"} Jun 06 22:12:07.053994 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.053970 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:07.084780 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.084754 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:07.163511 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.163478 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-bundle\") pod \"8f20015e-cd34-47d0-8330-de59913eed85\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " Jun 06 22:12:07.163511 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.163517 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwn9h\" (UniqueName: \"kubernetes.io/projected/d14f54af-3e4e-4f19-bce8-fea912d2f999-kube-api-access-lwn9h\") pod \"d14f54af-3e4e-4f19-bce8-fea912d2f999\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " Jun 06 22:12:07.163721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.163561 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-util\") pod \"8f20015e-cd34-47d0-8330-de59913eed85\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " Jun 06 22:12:07.163721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.163593 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-bundle\") pod \"d14f54af-3e4e-4f19-bce8-fea912d2f999\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " Jun 06 22:12:07.163721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.163613 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9glzt\" (UniqueName: \"kubernetes.io/projected/8f20015e-cd34-47d0-8330-de59913eed85-kube-api-access-9glzt\") pod \"8f20015e-cd34-47d0-8330-de59913eed85\" (UID: \"8f20015e-cd34-47d0-8330-de59913eed85\") " Jun 06 22:12:07.163721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.163663 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-util\") pod \"d14f54af-3e4e-4f19-bce8-fea912d2f999\" (UID: \"d14f54af-3e4e-4f19-bce8-fea912d2f999\") " Jun 06 22:12:07.164210 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.164180 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-bundle" (OuterVolumeSpecName: "bundle") pod "8f20015e-cd34-47d0-8330-de59913eed85" (UID: "8f20015e-cd34-47d0-8330-de59913eed85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:07.164491 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.164467 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-bundle" (OuterVolumeSpecName: "bundle") pod "d14f54af-3e4e-4f19-bce8-fea912d2f999" (UID: "d14f54af-3e4e-4f19-bce8-fea912d2f999"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:07.165800 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.165778 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d14f54af-3e4e-4f19-bce8-fea912d2f999-kube-api-access-lwn9h" (OuterVolumeSpecName: "kube-api-access-lwn9h") pod "d14f54af-3e4e-4f19-bce8-fea912d2f999" (UID: "d14f54af-3e4e-4f19-bce8-fea912d2f999"). InnerVolumeSpecName "kube-api-access-lwn9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:12:07.166209 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.166191 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f20015e-cd34-47d0-8330-de59913eed85-kube-api-access-9glzt" (OuterVolumeSpecName: "kube-api-access-9glzt") pod "8f20015e-cd34-47d0-8330-de59913eed85" (UID: "8f20015e-cd34-47d0-8330-de59913eed85"). InnerVolumeSpecName "kube-api-access-9glzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:12:07.169799 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.169760 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-util" (OuterVolumeSpecName: "util") pod "d14f54af-3e4e-4f19-bce8-fea912d2f999" (UID: "d14f54af-3e4e-4f19-bce8-fea912d2f999"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:07.169936 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.169916 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-util" (OuterVolumeSpecName: "util") pod "8f20015e-cd34-47d0-8330-de59913eed85" (UID: "8f20015e-cd34-47d0-8330-de59913eed85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:07.265331 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.265295 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:07.265331 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.265326 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwn9h\" (UniqueName: \"kubernetes.io/projected/d14f54af-3e4e-4f19-bce8-fea912d2f999-kube-api-access-lwn9h\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:07.265331 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.265335 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f20015e-cd34-47d0-8330-de59913eed85-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:07.265331 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.265344 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:07.265605 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.265352 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9glzt\" (UniqueName: \"kubernetes.io/projected/8f20015e-cd34-47d0-8330-de59913eed85-kube-api-access-9glzt\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:07.265605 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.265360 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d14f54af-3e4e-4f19-bce8-fea912d2f999-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:07.929765 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.929738 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" Jun 06 22:12:07.930168 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.929764 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65" event={"ID":"d14f54af-3e4e-4f19-bce8-fea912d2f999","Type":"ContainerDied","Data":"e74662da0a037352baed14246e24a8a6fc35108baa44291041711e5ba99de67e"} Jun 06 22:12:07.930168 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.929810 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e74662da0a037352baed14246e24a8a6fc35108baa44291041711e5ba99de67e" Jun 06 22:12:07.931774 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.931748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" event={"ID":"8f20015e-cd34-47d0-8330-de59913eed85","Type":"ContainerDied","Data":"5e89be458a4064b5d37c362992cd25c13a63da8394e93937e0e32de2aa3e654d"} Jun 06 22:12:07.931873 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.931780 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv" Jun 06 22:12:07.931873 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:07.931794 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e89be458a4064b5d37c362992cd25c13a63da8394e93937e0e32de2aa3e654d" Jun 06 22:12:08.063340 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.063314 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:08.174502 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.174471 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-util\") pod \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " Jun 06 22:12:08.174502 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.174506 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4rk7\" (UniqueName: \"kubernetes.io/projected/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-kube-api-access-g4rk7\") pod \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " Jun 06 22:12:08.174738 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.174549 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-bundle\") pod \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\" (UID: \"dfc51113-5723-4d8c-80f4-df8b6b7bb76f\") " Jun 06 22:12:08.175141 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.175080 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-bundle" (OuterVolumeSpecName: "bundle") pod "dfc51113-5723-4d8c-80f4-df8b6b7bb76f" (UID: "dfc51113-5723-4d8c-80f4-df8b6b7bb76f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:08.176696 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.176671 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-kube-api-access-g4rk7" (OuterVolumeSpecName: "kube-api-access-g4rk7") pod "dfc51113-5723-4d8c-80f4-df8b6b7bb76f" (UID: "dfc51113-5723-4d8c-80f4-df8b6b7bb76f"). InnerVolumeSpecName "kube-api-access-g4rk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:12:08.179751 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.179717 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-util" (OuterVolumeSpecName: "util") pod "dfc51113-5723-4d8c-80f4-df8b6b7bb76f" (UID: "dfc51113-5723-4d8c-80f4-df8b6b7bb76f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:12:08.275278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.275244 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:08.275278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.275273 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4rk7\" (UniqueName: \"kubernetes.io/projected/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-kube-api-access-g4rk7\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:08.275430 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.275286 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfc51113-5723-4d8c-80f4-df8b6b7bb76f-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:12:08.944437 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.944408 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" Jun 06 22:12:08.944884 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.944439 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr" event={"ID":"dfc51113-5723-4d8c-80f4-df8b6b7bb76f","Type":"ContainerDied","Data":"f697f11638a6f62236b4cff9885e69e3d599567c3976f371d04615b3e3254989"} Jun 06 22:12:08.944884 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:08.944521 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f697f11638a6f62236b4cff9885e69e3d599567c3976f371d04615b3e3254989" Jun 06 22:12:24.263913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.263875 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cb49bf655-qqznd"] Jun 06 22:12:24.264402 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264371 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="extract" Jun 06 22:12:24.264402 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264390 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="extract" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264404 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="extract" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264413 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="extract" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264426 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="pull" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264434 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="pull" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264446 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="extract" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264454 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="extract" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264471 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="util" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264479 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="util" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264488 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="util" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264496 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="util" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264504 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="extract" Jun 06 22:12:24.264518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264513 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="extract" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264526 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="util" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264534 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="util" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264551 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="util" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264559 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="util" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264567 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="pull" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264575 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="pull" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264584 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="pull" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264592 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="pull" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264607 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="pull" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264615 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="pull" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264698 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfc51113-5723-4d8c-80f4-df8b6b7bb76f" containerName="extract" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264711 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f20015e-cd34-47d0-8330-de59913eed85" containerName="extract" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264722 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c3a708e-c565-45bb-a322-dc7797186048" containerName="extract" Jun 06 22:12:24.265059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.264732 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d14f54af-3e4e-4f19-bce8-fea912d2f999" containerName="extract" Jun 06 22:12:24.267857 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.267839 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.281943 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.281921 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cb49bf655-qqznd"] Jun 06 22:12:24.417660 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417625 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-oauth-serving-cert\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.417798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417666 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-oauth-config\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.417798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-service-ca\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.417798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-serving-cert\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.417798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417773 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-trusted-ca-bundle\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.417798 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417789 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-config\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.417970 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.417806 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnbm4\" (UniqueName: \"kubernetes.io/projected/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-kube-api-access-bnbm4\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-oauth-serving-cert\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-oauth-config\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519289 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-service-ca\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519310 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-serving-cert\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-trusted-ca-bundle\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519467 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-config\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.519557 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.519487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnbm4\" (UniqueName: \"kubernetes.io/projected/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-kube-api-access-bnbm4\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.520143 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.520062 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-service-ca\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.520143 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.520062 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-oauth-serving-cert\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.520143 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.520090 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-config\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.520518 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.520493 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-trusted-ca-bundle\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.521871 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.521852 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-oauth-config\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.522082 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.522062 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-console-serving-cert\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.528133 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.528092 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnbm4\" (UniqueName: \"kubernetes.io/projected/4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf-kube-api-access-bnbm4\") pod \"console-6cb49bf655-qqznd\" (UID: \"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf\") " pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.577317 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.577286 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:24.727759 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:24.727733 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cb49bf655-qqznd"] Jun 06 22:12:24.729019 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:24.728996 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc9f7ed_53b9_4ae9_b9b5_4d45f7363adf.slice/crio-2a58ab7a533c1e52c59ba12ce0df0ae85a86622cf10bf853b61c61f253ee1c3e WatchSource:0}: Error finding container 2a58ab7a533c1e52c59ba12ce0df0ae85a86622cf10bf853b61c61f253ee1c3e: Status 404 returned error can't find the container with id 2a58ab7a533c1e52c59ba12ce0df0ae85a86622cf10bf853b61c61f253ee1c3e Jun 06 22:12:25.027697 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:25.027655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cb49bf655-qqznd" event={"ID":"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf","Type":"ContainerStarted","Data":"905a5cefaaf0f2cba0338e13cdbecac221f031ccba499ded168704e4a97110d1"} Jun 06 22:12:25.027844 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:25.027723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cb49bf655-qqznd" event={"ID":"4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf","Type":"ContainerStarted","Data":"2a58ab7a533c1e52c59ba12ce0df0ae85a86622cf10bf853b61c61f253ee1c3e"} Jun 06 22:12:25.051442 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:25.051336 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cb49bf655-qqznd" podStartSLOduration=1.051320384 podStartE2EDuration="1.051320384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:12:25.049236718 +0000 UTC m=+564.797801473" watchObservedRunningTime="2026-06-06 22:12:25.051320384 +0000 UTC m=+564.799885103" Jun 06 22:12:29.734909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.734868 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj"] Jun 06 22:12:29.738309 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.738294 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:29.741267 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.741250 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 06 22:12:29.742375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.742354 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 06 22:12:29.742499 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.742384 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 06 22:12:29.742499 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.742399 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dgvgp\"" Jun 06 22:12:29.750130 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.750088 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj"] Jun 06 22:12:29.867072 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.867042 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf2db\" (UniqueName: \"kubernetes.io/projected/1a7ccc57-fad0-4d9c-b081-17a1483c15ad-kube-api-access-rf2db\") pod \"dns-operator-controller-manager-864dd78c4c-sxtvj\" (UID: \"1a7ccc57-fad0-4d9c-b081-17a1483c15ad\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:29.968348 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.968319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf2db\" (UniqueName: \"kubernetes.io/projected/1a7ccc57-fad0-4d9c-b081-17a1483c15ad-kube-api-access-rf2db\") pod \"dns-operator-controller-manager-864dd78c4c-sxtvj\" (UID: \"1a7ccc57-fad0-4d9c-b081-17a1483c15ad\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:29.977690 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:29.977664 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf2db\" (UniqueName: \"kubernetes.io/projected/1a7ccc57-fad0-4d9c-b081-17a1483c15ad-kube-api-access-rf2db\") pod \"dns-operator-controller-manager-864dd78c4c-sxtvj\" (UID: \"1a7ccc57-fad0-4d9c-b081-17a1483c15ad\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:30.050083 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:30.050000 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:30.182037 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:30.182010 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj"] Jun 06 22:12:30.182980 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:30.182956 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7ccc57_fad0_4d9c_b081_17a1483c15ad.slice/crio-508c5021b5d6e95dbe5c6e11d0139734227d6e6efdad9a7f4e943624d8b728e8 WatchSource:0}: Error finding container 508c5021b5d6e95dbe5c6e11d0139734227d6e6efdad9a7f4e943624d8b728e8: Status 404 returned error can't find the container with id 508c5021b5d6e95dbe5c6e11d0139734227d6e6efdad9a7f4e943624d8b728e8 Jun 06 22:12:31.060157 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:31.060122 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" event={"ID":"1a7ccc57-fad0-4d9c-b081-17a1483c15ad","Type":"ContainerStarted","Data":"508c5021b5d6e95dbe5c6e11d0139734227d6e6efdad9a7f4e943624d8b728e8"} Jun 06 22:12:33.072263 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:33.072223 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" event={"ID":"1a7ccc57-fad0-4d9c-b081-17a1483c15ad","Type":"ContainerStarted","Data":"ae87a97fc804e26c6bb2c9f6c0744d118b1d1ed916818581a053c5948b9fdae4"} Jun 06 22:12:33.072655 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:33.072347 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:33.089887 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:33.089831 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" podStartSLOduration=1.700682035 podStartE2EDuration="4.089817238s" totalImagesPullingTime="2.389135203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:12:33.088964152 +0000 UTC m=+572.837528874" watchObservedRunningTime="2026-06-06 22:12:33.089817238 +0000 UTC m=+572.838381957" Jun 06 22:12:34.578148 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:34.578119 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:34.578664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:34.578173 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:34.582880 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:34.582861 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:35.086720 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.086688 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cb49bf655-qqznd" Jun 06 22:12:35.137948 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.137917 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b8b8f8dc-xjl24"] Jun 06 22:12:35.155357 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.155323 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh"] Jun 06 22:12:35.160030 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.160008 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.162531 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.162513 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-44xpd\"" Jun 06 22:12:35.162714 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.162694 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 06 22:12:35.162774 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.162738 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 06 22:12:35.167414 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.167393 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh"] Jun 06 22:12:35.323980 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.323946 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.324174 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.324008 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qbzp\" (UniqueName: \"kubernetes.io/projected/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-kube-api-access-8qbzp\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.324174 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.324050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.424993 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.424911 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qbzp\" (UniqueName: \"kubernetes.io/projected/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-kube-api-access-8qbzp\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.424993 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.424943 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.425242 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.424995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.425616 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.425594 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.427582 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.427561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.443443 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.443421 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qbzp\" (UniqueName: \"kubernetes.io/projected/1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c-kube-api-access-8qbzp\") pod \"kuadrant-console-plugin-66b5774985-vk2gh\" (UID: \"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.471305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.471281 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" Jun 06 22:12:35.600638 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:35.600606 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh"] Jun 06 22:12:35.601735 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:12:35.601707 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d1f8e1d_e46b_45bd_ac7a_62cbec57f05c.slice/crio-614c157e30496180be94eb66fbc1d6f7f6c199485a7c406fd8fa4acabd549bab WatchSource:0}: Error finding container 614c157e30496180be94eb66fbc1d6f7f6c199485a7c406fd8fa4acabd549bab: Status 404 returned error can't find the container with id 614c157e30496180be94eb66fbc1d6f7f6c199485a7c406fd8fa4acabd549bab Jun 06 22:12:36.089356 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:36.089314 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" event={"ID":"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c","Type":"ContainerStarted","Data":"614c157e30496180be94eb66fbc1d6f7f6c199485a7c406fd8fa4acabd549bab"} Jun 06 22:12:44.080369 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.080331 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sxtvj" Jun 06 22:12:44.780372 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.780313 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754"] Jun 06 22:12:44.801747 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.801708 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754"] Jun 06 22:12:44.801918 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.801843 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:44.804433 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.804405 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l7rcj\"" Jun 06 22:12:44.905519 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.905483 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl5sw\" (UniqueName: \"kubernetes.io/projected/d432892b-ff77-488b-8ba9-be980656413e-kube-api-access-dl5sw\") pod \"kuadrant-operator-controller-manager-55467d9f7-cb754\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:44.905701 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:44.905544 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d432892b-ff77-488b-8ba9-be980656413e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cb754\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:45.006531 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.006488 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl5sw\" (UniqueName: \"kubernetes.io/projected/d432892b-ff77-488b-8ba9-be980656413e-kube-api-access-dl5sw\") pod \"kuadrant-operator-controller-manager-55467d9f7-cb754\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:45.006729 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.006557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d432892b-ff77-488b-8ba9-be980656413e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cb754\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:45.007067 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.007037 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d432892b-ff77-488b-8ba9-be980656413e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cb754\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:45.015577 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.015554 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl5sw\" (UniqueName: \"kubernetes.io/projected/d432892b-ff77-488b-8ba9-be980656413e-kube-api-access-dl5sw\") pod \"kuadrant-operator-controller-manager-55467d9f7-cb754\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:45.037378 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.037313 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754"] Jun 06 22:12:45.037650 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.037635 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:12:45.053726 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:45.053699 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754"] Jun 06 22:12:57.232648 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:57.232614 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" event={"ID":"1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c","Type":"ContainerStarted","Data":"ca9626d132626d22322d1600c6e0c49af50a5d98211914f7a1ca4f48f48de003"} Jun 06 22:12:57.249180 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:12:57.249131 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vk2gh" podStartSLOduration=0.704965454 podStartE2EDuration="22.249110672s" totalImagesPullingTime="21.544145218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:12:57.24707975 +0000 UTC m=+596.995644470" watchObservedRunningTime="2026-06-06 22:12:57.249110672 +0000 UTC m=+596.997675393" Jun 06 22:12:59.607657 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:12:59.607611 2559 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-cb754_kuadrant-system_d432892b-ff77-488b-8ba9-be980656413e_0(134acdefc3d1871cd73b8b19f475fea3943aa0ce5527eb65b4cdc6e528182f82): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-cb754 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jun 06 22:12:59.607919 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:12:59.607683 2559 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-cb754_kuadrant-system_d432892b-ff77-488b-8ba9-be980656413e_0(134acdefc3d1871cd73b8b19f475fea3943aa0ce5527eb65b4cdc6e528182f82): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-cb754 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.158810 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.158764 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-65b8b8f8dc-xjl24" podUID="1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" containerName="console" containerID="cri-o://20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096" gracePeriod=15 Jun 06 22:13:00.248749 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.248722 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.250951 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:13:00.250921 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cb754\" is forbidden: User \"system:node:ip-10-0-136-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-77.ec2.internal' and this object" podUID="d432892b-ff77-488b-8ba9-be980656413e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.255299 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.255280 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.257444 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:13:00.257420 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cb754\" is forbidden: User \"system:node:ip-10-0-136-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-77.ec2.internal' and this object" podUID="d432892b-ff77-488b-8ba9-be980656413e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.348595 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.348564 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl5sw\" (UniqueName: \"kubernetes.io/projected/d432892b-ff77-488b-8ba9-be980656413e-kube-api-access-dl5sw\") pod \"d432892b-ff77-488b-8ba9-be980656413e\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " Jun 06 22:13:00.348740 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.348603 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d432892b-ff77-488b-8ba9-be980656413e-extensions-socket-volume\") pod \"d432892b-ff77-488b-8ba9-be980656413e\" (UID: \"d432892b-ff77-488b-8ba9-be980656413e\") " Jun 06 22:13:00.349032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.349003 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d432892b-ff77-488b-8ba9-be980656413e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d432892b-ff77-488b-8ba9-be980656413e" (UID: "d432892b-ff77-488b-8ba9-be980656413e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:13:00.350976 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.350941 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d432892b-ff77-488b-8ba9-be980656413e-kube-api-access-dl5sw" (OuterVolumeSpecName: "kube-api-access-dl5sw") pod "d432892b-ff77-488b-8ba9-be980656413e" (UID: "d432892b-ff77-488b-8ba9-be980656413e"). InnerVolumeSpecName "kube-api-access-dl5sw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:13:00.398666 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.398645 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:13:00.401090 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:13:00.401065 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cb754\" is forbidden: User \"system:node:ip-10-0-136-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-77.ec2.internal' and this object" podUID="d432892b-ff77-488b-8ba9-be980656413e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.449886 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.449860 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-serving-cert\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.449902 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-oauth-serving-cert\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.449933 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-trusted-ca-bundle\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450034 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.449962 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-oauth-config\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450231 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450067 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nmd5\" (UniqueName: \"kubernetes.io/projected/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-kube-api-access-8nmd5\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450231 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450182 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-service-ca\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450339 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450241 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-config\") pod \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\" (UID: \"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891\") " Jun 06 22:13:00.450400 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450343 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:13:00.450400 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450378 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:13:00.450568 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450545 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:13:00.450633 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450556 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-oauth-serving-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.450633 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450595 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-trusted-ca-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.450633 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450613 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl5sw\" (UniqueName: \"kubernetes.io/projected/d432892b-ff77-488b-8ba9-be980656413e-kube-api-access-dl5sw\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.450959 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450641 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d432892b-ff77-488b-8ba9-be980656413e-extensions-socket-volume\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.450959 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.450885 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-config" (OuterVolumeSpecName: "console-config") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:13:00.452346 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.452324 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:13:00.452447 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.452352 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:13:00.452447 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.452389 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-kube-api-access-8nmd5" (OuterVolumeSpecName: "kube-api-access-8nmd5") pod "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" (UID: "1ebbcbcc-a3b6-40ac-b136-12ab68d3a891"). InnerVolumeSpecName "kube-api-access-8nmd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:13:00.551139 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.551088 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-oauth-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.551139 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.551135 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nmd5\" (UniqueName: \"kubernetes.io/projected/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-kube-api-access-8nmd5\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.551313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.551148 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-service-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.551313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.551163 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-config\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.551313 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.551174 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891-console-serving-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:13:00.633087 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.633038 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:13:00.634118 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.634077 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:13:00.649786 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:13:00.649757 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cb754\" is forbidden: User \"system:node:ip-10-0-136-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-77.ec2.internal' and this object" podUID="d432892b-ff77-488b-8ba9-be980656413e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:00.650393 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:00.650373 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d432892b-ff77-488b-8ba9-be980656413e" path="/var/lib/kubelet/pods/d432892b-ff77-488b-8ba9-be980656413e/volumes" Jun 06 22:13:01.255338 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.255306 2559 generic.go:356] "Generic (PLEG): container finished" podID="1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" containerID="20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096" exitCode=0 Jun 06 22:13:01.255517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.255362 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b8b8f8dc-xjl24" Jun 06 22:13:01.255517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.255391 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b8b8f8dc-xjl24" event={"ID":"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891","Type":"ContainerDied","Data":"20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096"} Jun 06 22:13:01.255517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.255463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b8b8f8dc-xjl24" event={"ID":"1ebbcbcc-a3b6-40ac-b136-12ab68d3a891","Type":"ContainerDied","Data":"1894edf21ad1411073192383e1badd7f90302908f3bedec0fb96b673414074dd"} Jun 06 22:13:01.255517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.255484 2559 scope.go:117] "RemoveContainer" containerID="20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096" Jun 06 22:13:01.255667 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.255625 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:01.262439 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:13:01.262412 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cb754\" is forbidden: User \"system:node:ip-10-0-136-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-77.ec2.internal' and this object" podUID="d432892b-ff77-488b-8ba9-be980656413e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cb754" Jun 06 22:13:01.264472 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.264455 2559 scope.go:117] "RemoveContainer" containerID="20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096" Jun 06 22:13:01.264726 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:13:01.264706 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096\": container with ID starting with 20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096 not found: ID does not exist" containerID="20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096" Jun 06 22:13:01.264796 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.264738 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096"} err="failed to get container status \"20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096\": rpc error: code = NotFound desc = could not find container \"20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096\": container with ID starting with 20cab5b089162ff340ad3eb9832706b43ac0c75ed195bfa9e6b0582428c3c096 not found: ID does not exist" Jun 06 22:13:01.281851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.281831 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b8b8f8dc-xjl24"] Jun 06 22:13:01.287121 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:01.287075 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65b8b8f8dc-xjl24"] Jun 06 22:13:02.651135 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:02.651080 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" path="/var/lib/kubelet/pods/1ebbcbcc-a3b6-40ac-b136-12ab68d3a891/volumes" Jun 06 22:13:31.580395 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.580361 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:13:31.580792 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.580708 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" containerName="console" Jun 06 22:13:31.580792 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.580719 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" containerName="console" Jun 06 22:13:31.580792 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.580783 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ebbcbcc-a3b6-40ac-b136-12ab68d3a891" containerName="console" Jun 06 22:13:31.629299 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.629274 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:13:31.629299 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.629309 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:13:31.629535 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.629416 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.631846 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.631826 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 06 22:13:31.725417 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.725395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb69bb63-b06e-46ab-9dee-e94123fd109d-config-file\") pod \"limitador-limitador-54d6d988d5-8vkk8\" (UID: \"bb69bb63-b06e-46ab-9dee-e94123fd109d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.725540 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.725430 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnqhx\" (UniqueName: \"kubernetes.io/projected/bb69bb63-b06e-46ab-9dee-e94123fd109d-kube-api-access-jnqhx\") pod \"limitador-limitador-54d6d988d5-8vkk8\" (UID: \"bb69bb63-b06e-46ab-9dee-e94123fd109d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.826664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.826640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb69bb63-b06e-46ab-9dee-e94123fd109d-config-file\") pod \"limitador-limitador-54d6d988d5-8vkk8\" (UID: \"bb69bb63-b06e-46ab-9dee-e94123fd109d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.826752 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.826672 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnqhx\" (UniqueName: \"kubernetes.io/projected/bb69bb63-b06e-46ab-9dee-e94123fd109d-kube-api-access-jnqhx\") pod \"limitador-limitador-54d6d988d5-8vkk8\" (UID: \"bb69bb63-b06e-46ab-9dee-e94123fd109d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.827261 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.827242 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bb69bb63-b06e-46ab-9dee-e94123fd109d-config-file\") pod \"limitador-limitador-54d6d988d5-8vkk8\" (UID: \"bb69bb63-b06e-46ab-9dee-e94123fd109d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.834802 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.834756 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnqhx\" (UniqueName: \"kubernetes.io/projected/bb69bb63-b06e-46ab-9dee-e94123fd109d-kube-api-access-jnqhx\") pod \"limitador-limitador-54d6d988d5-8vkk8\" (UID: \"bb69bb63-b06e-46ab-9dee-e94123fd109d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:31.939835 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:31.939812 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:32.071793 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.071769 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:13:32.073110 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:13:32.073071 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb69bb63_b06e_46ab_9dee_e94123fd109d.slice/crio-9ac062a197f9df56c479406bd42688890001b47cee6467a0e114edd4efb60f17 WatchSource:0}: Error finding container 9ac062a197f9df56c479406bd42688890001b47cee6467a0e114edd4efb60f17: Status 404 returned error can't find the container with id 9ac062a197f9df56c479406bd42688890001b47cee6467a0e114edd4efb60f17 Jun 06 22:13:32.231963 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.231933 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mfjn5"] Jun 06 22:13:32.236550 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.236535 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:13:32.239001 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.238981 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rn25q\"" Jun 06 22:13:32.242229 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.242211 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mfjn5"] Jun 06 22:13:32.331205 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.331163 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq9ht\" (UniqueName: \"kubernetes.io/projected/0cf25132-becc-4c98-9bac-c8e64bda3ac3-kube-api-access-pq9ht\") pod \"authorino-75d556bd77-mfjn5\" (UID: \"0cf25132-becc-4c98-9bac-c8e64bda3ac3\") " pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:13:32.420254 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.420219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" event={"ID":"bb69bb63-b06e-46ab-9dee-e94123fd109d","Type":"ContainerStarted","Data":"9ac062a197f9df56c479406bd42688890001b47cee6467a0e114edd4efb60f17"} Jun 06 22:13:32.431581 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.431553 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq9ht\" (UniqueName: \"kubernetes.io/projected/0cf25132-becc-4c98-9bac-c8e64bda3ac3-kube-api-access-pq9ht\") pod \"authorino-75d556bd77-mfjn5\" (UID: \"0cf25132-becc-4c98-9bac-c8e64bda3ac3\") " pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:13:32.440065 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.440042 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq9ht\" (UniqueName: \"kubernetes.io/projected/0cf25132-becc-4c98-9bac-c8e64bda3ac3-kube-api-access-pq9ht\") pod \"authorino-75d556bd77-mfjn5\" (UID: \"0cf25132-becc-4c98-9bac-c8e64bda3ac3\") " pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:13:32.546537 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.546454 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:13:32.699528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:32.699494 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mfjn5"] Jun 06 22:13:32.701535 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:13:32.701506 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cf25132_becc_4c98_9bac_c8e64bda3ac3.slice/crio-f1d9c3edacd00aac262859a1656d00754cfb30ac42c54707b0515bbf574ff065 WatchSource:0}: Error finding container f1d9c3edacd00aac262859a1656d00754cfb30ac42c54707b0515bbf574ff065: Status 404 returned error can't find the container with id f1d9c3edacd00aac262859a1656d00754cfb30ac42c54707b0515bbf574ff065 Jun 06 22:13:33.432341 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:33.432262 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mfjn5" event={"ID":"0cf25132-becc-4c98-9bac-c8e64bda3ac3","Type":"ContainerStarted","Data":"f1d9c3edacd00aac262859a1656d00754cfb30ac42c54707b0515bbf574ff065"} Jun 06 22:13:37.458893 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:37.458847 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mfjn5" event={"ID":"0cf25132-becc-4c98-9bac-c8e64bda3ac3","Type":"ContainerStarted","Data":"1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64"} Jun 06 22:13:37.460399 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:37.460373 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" event={"ID":"bb69bb63-b06e-46ab-9dee-e94123fd109d","Type":"ContainerStarted","Data":"b34dd9f9e7b8c779344061ac8fa560fddaf96cd41eb933eba732266a7a41ac94"} Jun 06 22:13:37.460535 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:37.460433 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:13:37.476328 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:37.476264 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mfjn5" podStartSLOduration=1.7164431850000001 podStartE2EDuration="5.47625057s" totalImagesPullingTime="3.759807385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:13:37.473811386 +0000 UTC m=+637.222376105" watchObservedRunningTime="2026-06-06 22:13:37.47625057 +0000 UTC m=+637.224815288" Jun 06 22:13:37.490808 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:37.490769 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" podStartSLOduration=2.10958033 podStartE2EDuration="6.490755801s" totalImagesPullingTime="4.381175471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:13:37.48930739 +0000 UTC m=+637.237872128" watchObservedRunningTime="2026-06-06 22:13:37.490755801 +0000 UTC m=+637.239320518" Jun 06 22:13:48.466447 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:13:48.466415 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8vkk8" Jun 06 22:14:03.322479 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.322436 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt"] Jun 06 22:14:03.327385 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.327363 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.330325 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.330303 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 06 22:14:03.330464 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.330303 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 06 22:14:03.330464 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.330309 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7kgf8\"" Jun 06 22:14:03.336059 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.336030 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt"] Jun 06 22:14:03.486467 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.486417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.486664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.486545 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.486664 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.486573 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjh8\" (UniqueName: \"kubernetes.io/projected/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-kube-api-access-shjh8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.587991 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.587920 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.587991 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.587951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjh8\" (UniqueName: \"kubernetes.io/projected/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-kube-api-access-shjh8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.587991 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.587988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.588305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.588287 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.588347 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.588309 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.599677 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.599652 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjh8\" (UniqueName: \"kubernetes.io/projected/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-kube-api-access-shjh8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.638654 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.638627 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:03.777959 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:03.777934 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt"] Jun 06 22:14:03.779156 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:14:03.779121 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd0fd62_2f96_4e1d_866f_6a419a4ca0c9.slice/crio-75b3c62c77c4e41c17271aa9dba5ed3ae9ddee1d97d5c95a5f29176b644c356b WatchSource:0}: Error finding container 75b3c62c77c4e41c17271aa9dba5ed3ae9ddee1d97d5c95a5f29176b644c356b: Status 404 returned error can't find the container with id 75b3c62c77c4e41c17271aa9dba5ed3ae9ddee1d97d5c95a5f29176b644c356b Jun 06 22:14:04.604661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:04.604630 2559 generic.go:356] "Generic (PLEG): container finished" podID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerID="5d452955b0ae695e94d7025c826ffb77a3092d18755ab46722f4eb718c7ee376" exitCode=0 Jun 06 22:14:04.605129 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:04.604683 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" event={"ID":"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9","Type":"ContainerDied","Data":"5d452955b0ae695e94d7025c826ffb77a3092d18755ab46722f4eb718c7ee376"} Jun 06 22:14:04.605129 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:04.604744 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" event={"ID":"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9","Type":"ContainerStarted","Data":"75b3c62c77c4e41c17271aa9dba5ed3ae9ddee1d97d5c95a5f29176b644c356b"} Jun 06 22:14:05.612471 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:05.612442 2559 generic.go:356] "Generic (PLEG): container finished" podID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerID="fe7637e38ea8a656929655c82e5fe882e9843bcd97c58de58e64d0de5302f489" exitCode=0 Jun 06 22:14:05.612846 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:05.612534 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" event={"ID":"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9","Type":"ContainerDied","Data":"fe7637e38ea8a656929655c82e5fe882e9843bcd97c58de58e64d0de5302f489"} Jun 06 22:14:06.619873 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:06.619837 2559 generic.go:356] "Generic (PLEG): container finished" podID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerID="bb5d92f317d7d8f0726bf9d406670b44d0d177834784dadb9d70e439197308d2" exitCode=0 Jun 06 22:14:06.620274 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:06.619926 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" event={"ID":"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9","Type":"ContainerDied","Data":"bb5d92f317d7d8f0726bf9d406670b44d0d177834784dadb9d70e439197308d2"} Jun 06 22:14:07.765501 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.765475 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:07.924617 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.924540 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-util\") pod \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " Jun 06 22:14:07.924617 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.924596 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-bundle\") pod \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " Jun 06 22:14:07.924832 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.924638 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shjh8\" (UniqueName: \"kubernetes.io/projected/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-kube-api-access-shjh8\") pod \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\" (UID: \"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9\") " Jun 06 22:14:07.925142 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.925083 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-bundle" (OuterVolumeSpecName: "bundle") pod "1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" (UID: "1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:14:07.926721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.926697 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-kube-api-access-shjh8" (OuterVolumeSpecName: "kube-api-access-shjh8") pod "1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" (UID: "1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9"). InnerVolumeSpecName "kube-api-access-shjh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:14:07.930954 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:07.930934 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-util" (OuterVolumeSpecName: "util") pod "1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" (UID: "1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:14:08.026007 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:08.025979 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-bundle\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:14:08.026007 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:08.026005 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shjh8\" (UniqueName: \"kubernetes.io/projected/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-kube-api-access-shjh8\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:14:08.026207 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:08.026014 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9-util\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:14:08.639745 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:08.639717 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" Jun 06 22:14:08.639745 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:08.639723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748hwhvt" event={"ID":"1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9","Type":"ContainerDied","Data":"75b3c62c77c4e41c17271aa9dba5ed3ae9ddee1d97d5c95a5f29176b644c356b"} Jun 06 22:14:08.639951 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:08.639798 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75b3c62c77c4e41c17271aa9dba5ed3ae9ddee1d97d5c95a5f29176b644c356b" Jun 06 22:14:55.356304 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356221 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5j7gf"] Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356587 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="pull" Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356599 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="pull" Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356613 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="util" Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356619 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="util" Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356630 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="extract" Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356635 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="extract" Jun 06 22:14:55.356733 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.356690 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bd0fd62-2f96-4e1d-866f-6a419a4ca0c9" containerName="extract" Jun 06 22:14:55.359720 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.359705 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:55.370327 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.370304 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5j7gf"] Jun 06 22:14:55.403805 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.403782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4p5z\" (UniqueName: \"kubernetes.io/projected/a2bf46e3-3114-4621-a89b-24c1e83ee192-kube-api-access-g4p5z\") pod \"authorino-5d4b67c446-5j7gf\" (UID: \"a2bf46e3-3114-4621-a89b-24c1e83ee192\") " pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:55.504318 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.504286 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4p5z\" (UniqueName: \"kubernetes.io/projected/a2bf46e3-3114-4621-a89b-24c1e83ee192-kube-api-access-g4p5z\") pod \"authorino-5d4b67c446-5j7gf\" (UID: \"a2bf46e3-3114-4621-a89b-24c1e83ee192\") " pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:55.512158 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.512133 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4p5z\" (UniqueName: \"kubernetes.io/projected/a2bf46e3-3114-4621-a89b-24c1e83ee192-kube-api-access-g4p5z\") pod \"authorino-5d4b67c446-5j7gf\" (UID: \"a2bf46e3-3114-4621-a89b-24c1e83ee192\") " pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:55.567703 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.567675 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5j7gf"] Jun 06 22:14:55.567883 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.567871 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:55.596566 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.596543 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84fcbdd4b4-ktdb5"] Jun 06 22:14:55.600149 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.600133 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:55.606337 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.606279 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84fcbdd4b4-ktdb5"] Jun 06 22:14:55.702426 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.702398 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5j7gf"] Jun 06 22:14:55.703229 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:14:55.703204 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2bf46e3_3114_4621_a89b_24c1e83ee192.slice/crio-472e930c31cd6eabf83d9502af48785f3db0523aee2878cc3be9823da00b55b6 WatchSource:0}: Error finding container 472e930c31cd6eabf83d9502af48785f3db0523aee2878cc3be9823da00b55b6: Status 404 returned error can't find the container with id 472e930c31cd6eabf83d9502af48785f3db0523aee2878cc3be9823da00b55b6 Jun 06 22:14:55.704713 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.704694 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:14:55.705424 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.705402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85sgz\" (UniqueName: \"kubernetes.io/projected/2c6c82a0-f99a-4959-b880-96d4874dc46a-kube-api-access-85sgz\") pod \"authorino-84fcbdd4b4-ktdb5\" (UID: \"2c6c82a0-f99a-4959-b880-96d4874dc46a\") " pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:55.806171 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.806142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85sgz\" (UniqueName: \"kubernetes.io/projected/2c6c82a0-f99a-4959-b880-96d4874dc46a-kube-api-access-85sgz\") pod \"authorino-84fcbdd4b4-ktdb5\" (UID: \"2c6c82a0-f99a-4959-b880-96d4874dc46a\") " pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:55.815593 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.815568 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85sgz\" (UniqueName: \"kubernetes.io/projected/2c6c82a0-f99a-4959-b880-96d4874dc46a-kube-api-access-85sgz\") pod \"authorino-84fcbdd4b4-ktdb5\" (UID: \"2c6c82a0-f99a-4959-b880-96d4874dc46a\") " pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:55.835592 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.835567 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84fcbdd4b4-ktdb5"] Jun 06 22:14:55.835810 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.835798 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:55.859274 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.859214 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84b6d8cd48-jcrrt"] Jun 06 22:14:55.865810 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.864624 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:55.867606 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.867554 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 06 22:14:55.871706 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.871680 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84b6d8cd48-jcrrt"] Jun 06 22:14:55.899628 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.899600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" event={"ID":"a2bf46e3-3114-4621-a89b-24c1e83ee192","Type":"ContainerStarted","Data":"472e930c31cd6eabf83d9502af48785f3db0523aee2878cc3be9823da00b55b6"} Jun 06 22:14:55.907569 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.907538 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3aebe24e-b69e-4042-aa68-bae966944bf2-tls-cert\") pod \"authorino-84b6d8cd48-jcrrt\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:55.907663 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.907579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55svc\" (UniqueName: \"kubernetes.io/projected/3aebe24e-b69e-4042-aa68-bae966944bf2-kube-api-access-55svc\") pod \"authorino-84b6d8cd48-jcrrt\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:55.967923 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:55.967894 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84fcbdd4b4-ktdb5"] Jun 06 22:14:55.968850 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:14:55.968821 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c6c82a0_f99a_4959_b880_96d4874dc46a.slice/crio-c3c5f78c850815fa9847351f47dc8b3ebc86447e1a702d8998cb642b5eb25967 WatchSource:0}: Error finding container c3c5f78c850815fa9847351f47dc8b3ebc86447e1a702d8998cb642b5eb25967: Status 404 returned error can't find the container with id c3c5f78c850815fa9847351f47dc8b3ebc86447e1a702d8998cb642b5eb25967 Jun 06 22:14:56.008992 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.008971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3aebe24e-b69e-4042-aa68-bae966944bf2-tls-cert\") pod \"authorino-84b6d8cd48-jcrrt\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:56.009078 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.009003 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55svc\" (UniqueName: \"kubernetes.io/projected/3aebe24e-b69e-4042-aa68-bae966944bf2-kube-api-access-55svc\") pod \"authorino-84b6d8cd48-jcrrt\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:56.011327 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.011303 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3aebe24e-b69e-4042-aa68-bae966944bf2-tls-cert\") pod \"authorino-84b6d8cd48-jcrrt\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:56.017227 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.017202 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55svc\" (UniqueName: \"kubernetes.io/projected/3aebe24e-b69e-4042-aa68-bae966944bf2-kube-api-access-55svc\") pod \"authorino-84b6d8cd48-jcrrt\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:56.177863 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.177825 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:14:56.310070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.310048 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84b6d8cd48-jcrrt"] Jun 06 22:14:56.316192 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:14:56.316169 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aebe24e_b69e_4042_aa68_bae966944bf2.slice/crio-81f22fe1544ca1f92e2424ca2f5129a3d1f5f99ef6d13b0ed0a83fea0070ef01 WatchSource:0}: Error finding container 81f22fe1544ca1f92e2424ca2f5129a3d1f5f99ef6d13b0ed0a83fea0070ef01: Status 404 returned error can't find the container with id 81f22fe1544ca1f92e2424ca2f5129a3d1f5f99ef6d13b0ed0a83fea0070ef01 Jun 06 22:14:56.906840 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.906748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" event={"ID":"3aebe24e-b69e-4042-aa68-bae966944bf2","Type":"ContainerStarted","Data":"337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515"} Jun 06 22:14:56.906840 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.906795 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" event={"ID":"3aebe24e-b69e-4042-aa68-bae966944bf2","Type":"ContainerStarted","Data":"81f22fe1544ca1f92e2424ca2f5129a3d1f5f99ef6d13b0ed0a83fea0070ef01"} Jun 06 22:14:56.908336 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.908312 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" event={"ID":"a2bf46e3-3114-4621-a89b-24c1e83ee192","Type":"ContainerStarted","Data":"4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60"} Jun 06 22:14:56.908336 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.908314 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" podUID="a2bf46e3-3114-4621-a89b-24c1e83ee192" containerName="authorino" containerID="cri-o://4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60" gracePeriod=30 Jun 06 22:14:56.910140 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.910092 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" event={"ID":"2c6c82a0-f99a-4959-b880-96d4874dc46a","Type":"ContainerStarted","Data":"aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5"} Jun 06 22:14:56.910244 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.910149 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" event={"ID":"2c6c82a0-f99a-4959-b880-96d4874dc46a","Type":"ContainerStarted","Data":"c3c5f78c850815fa9847351f47dc8b3ebc86447e1a702d8998cb642b5eb25967"} Jun 06 22:14:56.910244 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.910198 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" podUID="2c6c82a0-f99a-4959-b880-96d4874dc46a" containerName="authorino" containerID="cri-o://aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5" gracePeriod=30 Jun 06 22:14:56.924170 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.924130 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" podStartSLOduration=1.638310602 podStartE2EDuration="1.924116576s" totalImagesPullingTime="285.805974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:14:56.92072857 +0000 UTC m=+716.669293289" watchObservedRunningTime="2026-06-06 22:14:56.924116576 +0000 UTC m=+716.672681294" Jun 06 22:14:56.938347 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.938306 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" podStartSLOduration=1.624837289 podStartE2EDuration="1.938293762s" totalImagesPullingTime="313.456473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:14:56.934551319 +0000 UTC m=+716.683116042" watchObservedRunningTime="2026-06-06 22:14:56.938293762 +0000 UTC m=+716.686858459" Jun 06 22:14:56.963818 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.963790 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mfjn5"] Jun 06 22:14:56.963975 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.963954 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mfjn5" podUID="0cf25132-becc-4c98-9bac-c8e64bda3ac3" containerName="authorino" containerID="cri-o://1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64" gracePeriod=30 Jun 06 22:14:56.966070 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:56.966021 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" podStartSLOduration=1.6184504 podStartE2EDuration="1.966004372s" totalImagesPullingTime="347.553972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:14:56.963316795 +0000 UTC m=+716.711881505" watchObservedRunningTime="2026-06-06 22:14:56.966004372 +0000 UTC m=+716.714569097" Jun 06 22:14:57.230291 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.230259 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:57.239852 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.239831 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:57.265494 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.265470 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:14:57.322805 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.322772 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq9ht\" (UniqueName: \"kubernetes.io/projected/0cf25132-becc-4c98-9bac-c8e64bda3ac3-kube-api-access-pq9ht\") pod \"0cf25132-becc-4c98-9bac-c8e64bda3ac3\" (UID: \"0cf25132-becc-4c98-9bac-c8e64bda3ac3\") " Jun 06 22:14:57.322960 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.322837 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85sgz\" (UniqueName: \"kubernetes.io/projected/2c6c82a0-f99a-4959-b880-96d4874dc46a-kube-api-access-85sgz\") pod \"2c6c82a0-f99a-4959-b880-96d4874dc46a\" (UID: \"2c6c82a0-f99a-4959-b880-96d4874dc46a\") " Jun 06 22:14:57.322960 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.322885 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4p5z\" (UniqueName: \"kubernetes.io/projected/a2bf46e3-3114-4621-a89b-24c1e83ee192-kube-api-access-g4p5z\") pod \"a2bf46e3-3114-4621-a89b-24c1e83ee192\" (UID: \"a2bf46e3-3114-4621-a89b-24c1e83ee192\") " Jun 06 22:14:57.325019 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.324993 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf25132-becc-4c98-9bac-c8e64bda3ac3-kube-api-access-pq9ht" (OuterVolumeSpecName: "kube-api-access-pq9ht") pod "0cf25132-becc-4c98-9bac-c8e64bda3ac3" (UID: "0cf25132-becc-4c98-9bac-c8e64bda3ac3"). InnerVolumeSpecName "kube-api-access-pq9ht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:14:57.325019 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.325001 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c6c82a0-f99a-4959-b880-96d4874dc46a-kube-api-access-85sgz" (OuterVolumeSpecName: "kube-api-access-85sgz") pod "2c6c82a0-f99a-4959-b880-96d4874dc46a" (UID: "2c6c82a0-f99a-4959-b880-96d4874dc46a"). InnerVolumeSpecName "kube-api-access-85sgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:14:57.325019 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.325006 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2bf46e3-3114-4621-a89b-24c1e83ee192-kube-api-access-g4p5z" (OuterVolumeSpecName: "kube-api-access-g4p5z") pod "a2bf46e3-3114-4621-a89b-24c1e83ee192" (UID: "a2bf46e3-3114-4621-a89b-24c1e83ee192"). InnerVolumeSpecName "kube-api-access-g4p5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:14:57.423516 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.423487 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4p5z\" (UniqueName: \"kubernetes.io/projected/a2bf46e3-3114-4621-a89b-24c1e83ee192-kube-api-access-g4p5z\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:14:57.423516 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.423510 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq9ht\" (UniqueName: \"kubernetes.io/projected/0cf25132-becc-4c98-9bac-c8e64bda3ac3-kube-api-access-pq9ht\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:14:57.423516 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.423519 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85sgz\" (UniqueName: \"kubernetes.io/projected/2c6c82a0-f99a-4959-b880-96d4874dc46a-kube-api-access-85sgz\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:14:57.916898 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.916813 2559 generic.go:356] "Generic (PLEG): container finished" podID="a2bf46e3-3114-4621-a89b-24c1e83ee192" containerID="4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60" exitCode=0 Jun 06 22:14:57.916898 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.916863 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" Jun 06 22:14:57.917382 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.916898 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" event={"ID":"a2bf46e3-3114-4621-a89b-24c1e83ee192","Type":"ContainerDied","Data":"4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60"} Jun 06 22:14:57.917382 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.916937 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5j7gf" event={"ID":"a2bf46e3-3114-4621-a89b-24c1e83ee192","Type":"ContainerDied","Data":"472e930c31cd6eabf83d9502af48785f3db0523aee2878cc3be9823da00b55b6"} Jun 06 22:14:57.917382 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.916956 2559 scope.go:117] "RemoveContainer" containerID="4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60" Jun 06 22:14:57.918288 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.918237 2559 generic.go:356] "Generic (PLEG): container finished" podID="0cf25132-becc-4c98-9bac-c8e64bda3ac3" containerID="1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64" exitCode=0 Jun 06 22:14:57.918288 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.918272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mfjn5" event={"ID":"0cf25132-becc-4c98-9bac-c8e64bda3ac3","Type":"ContainerDied","Data":"1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64"} Jun 06 22:14:57.918426 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.918303 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mfjn5" event={"ID":"0cf25132-becc-4c98-9bac-c8e64bda3ac3","Type":"ContainerDied","Data":"f1d9c3edacd00aac262859a1656d00754cfb30ac42c54707b0515bbf574ff065"} Jun 06 22:14:57.918426 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.918277 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mfjn5" Jun 06 22:14:57.919693 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.919671 2559 generic.go:356] "Generic (PLEG): container finished" podID="2c6c82a0-f99a-4959-b880-96d4874dc46a" containerID="aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5" exitCode=0 Jun 06 22:14:57.919806 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.919702 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" Jun 06 22:14:57.919806 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.919764 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" event={"ID":"2c6c82a0-f99a-4959-b880-96d4874dc46a","Type":"ContainerDied","Data":"aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5"} Jun 06 22:14:57.919923 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.919799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84fcbdd4b4-ktdb5" event={"ID":"2c6c82a0-f99a-4959-b880-96d4874dc46a","Type":"ContainerDied","Data":"c3c5f78c850815fa9847351f47dc8b3ebc86447e1a702d8998cb642b5eb25967"} Jun 06 22:14:57.926797 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.926780 2559 scope.go:117] "RemoveContainer" containerID="4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60" Jun 06 22:14:57.927198 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:14:57.927057 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60\": container with ID starting with 4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60 not found: ID does not exist" containerID="4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60" Jun 06 22:14:57.927198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.927140 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60"} err="failed to get container status \"4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60\": rpc error: code = NotFound desc = could not find container \"4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60\": container with ID starting with 4c5af4c6fa0de7d8253f68c9207a0d031398df34d25b097783febc814e8b2e60 not found: ID does not exist" Jun 06 22:14:57.927198 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.927188 2559 scope.go:117] "RemoveContainer" containerID="1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64" Jun 06 22:14:57.935206 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.935189 2559 scope.go:117] "RemoveContainer" containerID="1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64" Jun 06 22:14:57.935478 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:14:57.935461 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64\": container with ID starting with 1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64 not found: ID does not exist" containerID="1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64" Jun 06 22:14:57.935538 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.935484 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64"} err="failed to get container status \"1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64\": rpc error: code = NotFound desc = could not find container \"1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64\": container with ID starting with 1e34871dddde8ec153cbab8d72d328dbdd9f78b3515637a5e1b9ecefa9103c64 not found: ID does not exist" Jun 06 22:14:57.935538 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.935499 2559 scope.go:117] "RemoveContainer" containerID="aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5" Jun 06 22:14:57.943559 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.943542 2559 scope.go:117] "RemoveContainer" containerID="aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5" Jun 06 22:14:57.943637 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.943541 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mfjn5"] Jun 06 22:14:57.943839 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:14:57.943818 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5\": container with ID starting with aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5 not found: ID does not exist" containerID="aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5" Jun 06 22:14:57.943879 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.943849 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5"} err="failed to get container status \"aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5\": rpc error: code = NotFound desc = could not find container \"aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5\": container with ID starting with aec3cc02aa40c619e28685d2fe6423a485591b99ac407886162e6dfdcaa998a5 not found: ID does not exist" Jun 06 22:14:57.945956 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.945940 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mfjn5"] Jun 06 22:14:57.955333 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.955314 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5j7gf"] Jun 06 22:14:57.961575 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.961554 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5j7gf"] Jun 06 22:14:57.970812 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.970788 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84fcbdd4b4-ktdb5"] Jun 06 22:14:57.976329 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:57.976310 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84fcbdd4b4-ktdb5"] Jun 06 22:14:58.657926 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:58.657893 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf25132-becc-4c98-9bac-c8e64bda3ac3" path="/var/lib/kubelet/pods/0cf25132-becc-4c98-9bac-c8e64bda3ac3/volumes" Jun 06 22:14:58.658228 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:58.658213 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c6c82a0-f99a-4959-b880-96d4874dc46a" path="/var/lib/kubelet/pods/2c6c82a0-f99a-4959-b880-96d4874dc46a/volumes" Jun 06 22:14:58.658489 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:14:58.658477 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2bf46e3-3114-4621-a89b-24c1e83ee192" path="/var/lib/kubelet/pods/a2bf46e3-3114-4621-a89b-24c1e83ee192/volumes" Jun 06 22:15:37.283201 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:15:37.283165 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:16:34.983761 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.983719 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b64bbf866-xqvfg"] Jun 06 22:16:34.984375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984288 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf25132-becc-4c98-9bac-c8e64bda3ac3" containerName="authorino" Jun 06 22:16:34.984375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984309 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf25132-becc-4c98-9bac-c8e64bda3ac3" containerName="authorino" Jun 06 22:16:34.984375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984321 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2bf46e3-3114-4621-a89b-24c1e83ee192" containerName="authorino" Jun 06 22:16:34.984375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984330 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2bf46e3-3114-4621-a89b-24c1e83ee192" containerName="authorino" Jun 06 22:16:34.984375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984352 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c6c82a0-f99a-4959-b880-96d4874dc46a" containerName="authorino" Jun 06 22:16:34.984375 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984360 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c6c82a0-f99a-4959-b880-96d4874dc46a" containerName="authorino" Jun 06 22:16:34.984680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984441 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2bf46e3-3114-4621-a89b-24c1e83ee192" containerName="authorino" Jun 06 22:16:34.984680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984459 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf25132-becc-4c98-9bac-c8e64bda3ac3" containerName="authorino" Jun 06 22:16:34.984680 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.984468 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c6c82a0-f99a-4959-b880-96d4874dc46a" containerName="authorino" Jun 06 22:16:34.987480 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.987461 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:34.989902 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.989884 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 06 22:16:34.994944 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:34.994920 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b64bbf866-xqvfg"] Jun 06 22:16:35.051860 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.051813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/23450e96-beda-4410-8d5f-06e9ebd88a3b-oidc-ca\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.052033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.051863 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckn86\" (UniqueName: \"kubernetes.io/projected/23450e96-beda-4410-8d5f-06e9ebd88a3b-kube-api-access-ckn86\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.052033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.051942 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23450e96-beda-4410-8d5f-06e9ebd88a3b-tls-cert\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.153287 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.153258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/23450e96-beda-4410-8d5f-06e9ebd88a3b-oidc-ca\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.153287 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.153290 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckn86\" (UniqueName: \"kubernetes.io/projected/23450e96-beda-4410-8d5f-06e9ebd88a3b-kube-api-access-ckn86\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.153507 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.153438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23450e96-beda-4410-8d5f-06e9ebd88a3b-tls-cert\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.153851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.153829 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/23450e96-beda-4410-8d5f-06e9ebd88a3b-oidc-ca\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.155909 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.155889 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23450e96-beda-4410-8d5f-06e9ebd88a3b-tls-cert\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.160621 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.160596 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckn86\" (UniqueName: \"kubernetes.io/projected/23450e96-beda-4410-8d5f-06e9ebd88a3b-kube-api-access-ckn86\") pod \"authorino-7b64bbf866-xqvfg\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.297012 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.296907 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:16:35.430673 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.430646 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b64bbf866-xqvfg"] Jun 06 22:16:35.432370 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:16:35.432343 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23450e96_beda_4410_8d5f_06e9ebd88a3b.slice/crio-a21267e1a68ead20d83d40d1e6a3c42f5f1d3c3967b7c9319c1203e78613e313 WatchSource:0}: Error finding container a21267e1a68ead20d83d40d1e6a3c42f5f1d3c3967b7c9319c1203e78613e313: Status 404 returned error can't find the container with id a21267e1a68ead20d83d40d1e6a3c42f5f1d3c3967b7c9319c1203e78613e313 Jun 06 22:16:35.464377 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:35.464348 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" event={"ID":"23450e96-beda-4410-8d5f-06e9ebd88a3b","Type":"ContainerStarted","Data":"a21267e1a68ead20d83d40d1e6a3c42f5f1d3c3967b7c9319c1203e78613e313"} Jun 06 22:16:36.472509 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.472468 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" event={"ID":"23450e96-beda-4410-8d5f-06e9ebd88a3b","Type":"ContainerStarted","Data":"20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf"} Jun 06 22:16:36.492114 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.492044 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" podStartSLOduration=2.093209437 podStartE2EDuration="2.492026991s" totalImagesPullingTime="398.817554ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:16:36.489286057 +0000 UTC m=+816.237850776" watchObservedRunningTime="2026-06-06 22:16:36.492026991 +0000 UTC m=+816.240591730" Jun 06 22:16:36.516670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.516636 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84b6d8cd48-jcrrt"] Jun 06 22:16:36.516877 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.516836 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" podUID="3aebe24e-b69e-4042-aa68-bae966944bf2" containerName="authorino" containerID="cri-o://337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515" gracePeriod=30 Jun 06 22:16:36.777412 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.777384 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:16:36.869409 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.869381 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55svc\" (UniqueName: \"kubernetes.io/projected/3aebe24e-b69e-4042-aa68-bae966944bf2-kube-api-access-55svc\") pod \"3aebe24e-b69e-4042-aa68-bae966944bf2\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " Jun 06 22:16:36.869599 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.869468 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3aebe24e-b69e-4042-aa68-bae966944bf2-tls-cert\") pod \"3aebe24e-b69e-4042-aa68-bae966944bf2\" (UID: \"3aebe24e-b69e-4042-aa68-bae966944bf2\") " Jun 06 22:16:36.871632 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.871597 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aebe24e-b69e-4042-aa68-bae966944bf2-kube-api-access-55svc" (OuterVolumeSpecName: "kube-api-access-55svc") pod "3aebe24e-b69e-4042-aa68-bae966944bf2" (UID: "3aebe24e-b69e-4042-aa68-bae966944bf2"). InnerVolumeSpecName "kube-api-access-55svc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:16:36.879620 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.879595 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3aebe24e-b69e-4042-aa68-bae966944bf2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3aebe24e-b69e-4042-aa68-bae966944bf2" (UID: "3aebe24e-b69e-4042-aa68-bae966944bf2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:16:36.970342 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.970300 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55svc\" (UniqueName: \"kubernetes.io/projected/3aebe24e-b69e-4042-aa68-bae966944bf2-kube-api-access-55svc\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:16:36.970342 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:36.970339 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3aebe24e-b69e-4042-aa68-bae966944bf2-tls-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:16:37.480092 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.480052 2559 generic.go:356] "Generic (PLEG): container finished" podID="3aebe24e-b69e-4042-aa68-bae966944bf2" containerID="337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515" exitCode=0 Jun 06 22:16:37.480562 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.480130 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" Jun 06 22:16:37.480562 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.480151 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" event={"ID":"3aebe24e-b69e-4042-aa68-bae966944bf2","Type":"ContainerDied","Data":"337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515"} Jun 06 22:16:37.480562 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.480199 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84b6d8cd48-jcrrt" event={"ID":"3aebe24e-b69e-4042-aa68-bae966944bf2","Type":"ContainerDied","Data":"81f22fe1544ca1f92e2424ca2f5129a3d1f5f99ef6d13b0ed0a83fea0070ef01"} Jun 06 22:16:37.480562 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.480219 2559 scope.go:117] "RemoveContainer" containerID="337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515" Jun 06 22:16:37.492403 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.492381 2559 scope.go:117] "RemoveContainer" containerID="337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515" Jun 06 22:16:37.492727 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:16:37.492706 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515\": container with ID starting with 337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515 not found: ID does not exist" containerID="337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515" Jun 06 22:16:37.492799 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.492740 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515"} err="failed to get container status \"337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515\": rpc error: code = NotFound desc = could not find container \"337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515\": container with ID starting with 337fd966cd7371617dc394a56fa517ac9b5c793aefaabd88ba480644865f3515 not found: ID does not exist" Jun 06 22:16:37.504440 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.504419 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84b6d8cd48-jcrrt"] Jun 06 22:16:37.509934 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:37.509914 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84b6d8cd48-jcrrt"] Jun 06 22:16:38.650414 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:38.650380 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aebe24e-b69e-4042-aa68-bae966944bf2" path="/var/lib/kubelet/pods/3aebe24e-b69e-4042-aa68-bae966944bf2/volumes" Jun 06 22:16:48.444670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:48.444627 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:16:58.948704 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.948661 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h"] Jun 06 22:16:58.949231 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.949212 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aebe24e-b69e-4042-aa68-bae966944bf2" containerName="authorino" Jun 06 22:16:58.949320 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.949232 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aebe24e-b69e-4042-aa68-bae966944bf2" containerName="authorino" Jun 06 22:16:58.949384 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.949337 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aebe24e-b69e-4042-aa68-bae966944bf2" containerName="authorino" Jun 06 22:16:58.952661 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.952642 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:58.955217 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.955177 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 06 22:16:58.955217 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.955213 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 06 22:16:58.956354 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.956338 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 06 22:16:58.956435 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.956389 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mh76x\"" Jun 06 22:16:58.964271 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:58.964251 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h"] Jun 06 22:16:59.054624 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.054577 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.054816 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.054647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.054816 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.054694 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.054816 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.054721 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.054816 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.054746 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4kg\" (UniqueName: \"kubernetes.io/projected/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-kube-api-access-sm4kg\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.055000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.054834 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.155838 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.155801 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.155981 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.155858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.155981 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.155906 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.155981 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.155939 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.155981 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.155967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.156179 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.155991 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4kg\" (UniqueName: \"kubernetes.io/projected/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-kube-api-access-sm4kg\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.156422 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.156387 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.156422 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.156408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.156576 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.156474 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.158299 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.158269 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.158542 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.158525 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.163648 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.163625 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4kg\" (UniqueName: \"kubernetes.io/projected/fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482-kube-api-access-sm4kg\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-22z9h\" (UID: \"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.262962 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.262865 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:16:59.363630 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.363597 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:16:59.398824 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.398794 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h"] Jun 06 22:16:59.399916 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:16:59.399891 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1ed1e3_6e4f_4b9f_8e39_8d24930cd482.slice/crio-2241af82f933f30bb5aecd10706c6b7a14c6d42c8b7c110342fae0983aae1e99 WatchSource:0}: Error finding container 2241af82f933f30bb5aecd10706c6b7a14c6d42c8b7c110342fae0983aae1e99: Status 404 returned error can't find the container with id 2241af82f933f30bb5aecd10706c6b7a14c6d42c8b7c110342fae0983aae1e99 Jun 06 22:16:59.609411 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:16:59.609327 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" event={"ID":"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482","Type":"ContainerStarted","Data":"2241af82f933f30bb5aecd10706c6b7a14c6d42c8b7c110342fae0983aae1e99"} Jun 06 22:17:01.951517 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:01.951484 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:17:02.631032 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:02.630940 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" event={"ID":"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482","Type":"ContainerStarted","Data":"0b86e4a2a08bb4d1db6c97b83ac8f06ec4bf8564ef2e7196ca5ff470d14499b9"} Jun 06 22:17:03.638155 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:03.638122 2559 generic.go:356] "Generic (PLEG): container finished" podID="fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482" containerID="0b86e4a2a08bb4d1db6c97b83ac8f06ec4bf8564ef2e7196ca5ff470d14499b9" exitCode=0 Jun 06 22:17:03.638581 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:03.638224 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" event={"ID":"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482","Type":"ContainerDied","Data":"0b86e4a2a08bb4d1db6c97b83ac8f06ec4bf8564ef2e7196ca5ff470d14499b9"} Jun 06 22:17:05.652273 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:05.652239 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" event={"ID":"fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482","Type":"ContainerStarted","Data":"076e8d77f1aee3ba8b849214ad6b63e4ba2bdc6851cc790866a0256317582b7d"} Jun 06 22:17:05.673902 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:05.673844 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" podStartSLOduration=2.544078108 podStartE2EDuration="7.673827718s" totalImagesPullingTime="3.813476923s" totalInitContainerRuntime="1.316272687s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:16:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:17:05.671953049 +0000 UTC m=+845.420517788" watchObservedRunningTime="2026-06-06 22:17:05.673827718 +0000 UTC m=+845.422392486" Jun 06 22:17:09.263887 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:09.263849 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:17:09.264393 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:09.264006 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:17:09.278293 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:09.278271 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:17:09.685725 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:09.685651 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-22z9h" Jun 06 22:17:22.960898 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:22.960863 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:17:34.860145 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:34.860082 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:17:41.062506 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.062474 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp"] Jun 06 22:17:41.066335 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.066313 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.070015 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.069986 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 06 22:17:41.078081 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.078047 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp"] Jun 06 22:17:41.119289 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.119259 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.119432 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.119295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.119432 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.119327 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a67deb-0691-4511-b6cb-dab9222d48b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.119432 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.119353 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqwvd\" (UniqueName: \"kubernetes.io/projected/20a67deb-0691-4511-b6cb-dab9222d48b0-kube-api-access-cqwvd\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.119432 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.119400 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.119432 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.119415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220362 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220322 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a67deb-0691-4511-b6cb-dab9222d48b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220362 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqwvd\" (UniqueName: \"kubernetes.io/projected/20a67deb-0691-4511-b6cb-dab9222d48b0-kube-api-access-cqwvd\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220565 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220407 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220565 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220565 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220579 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.220928 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220908 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.221033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.220973 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.221033 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.221001 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.222902 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.222878 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a67deb-0691-4511-b6cb-dab9222d48b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.223055 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.223038 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a67deb-0691-4511-b6cb-dab9222d48b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.227719 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.227701 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqwvd\" (UniqueName: \"kubernetes.io/projected/20a67deb-0691-4511-b6cb-dab9222d48b0-kube-api-access-cqwvd\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-njmlp\" (UID: \"20a67deb-0691-4511-b6cb-dab9222d48b0\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.405842 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.405763 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:41.539413 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.539387 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp"] Jun 06 22:17:41.541209 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:17:41.541184 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a67deb_0691_4511_b6cb_dab9222d48b0.slice/crio-6a9b37377d2fc069bff1aa87609db62f65a5245ece2ba4edf1a092b73691c6d1 WatchSource:0}: Error finding container 6a9b37377d2fc069bff1aa87609db62f65a5245ece2ba4edf1a092b73691c6d1: Status 404 returned error can't find the container with id 6a9b37377d2fc069bff1aa87609db62f65a5245ece2ba4edf1a092b73691c6d1 Jun 06 22:17:41.857177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.857135 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" event={"ID":"20a67deb-0691-4511-b6cb-dab9222d48b0","Type":"ContainerStarted","Data":"a9df4f962f896e3d9364613255710c5e38f442b4d3d86d54c35c9bcdfb2ede24"} Jun 06 22:17:41.857177 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.857185 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" event={"ID":"20a67deb-0691-4511-b6cb-dab9222d48b0","Type":"ContainerStarted","Data":"6a9b37377d2fc069bff1aa87609db62f65a5245ece2ba4edf1a092b73691c6d1"} Jun 06 22:17:41.956249 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:41.956209 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:17:42.863862 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:42.863835 2559 generic.go:356] "Generic (PLEG): container finished" podID="20a67deb-0691-4511-b6cb-dab9222d48b0" containerID="a9df4f962f896e3d9364613255710c5e38f442b4d3d86d54c35c9bcdfb2ede24" exitCode=0 Jun 06 22:17:42.864178 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:42.863925 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" event={"ID":"20a67deb-0691-4511-b6cb-dab9222d48b0","Type":"ContainerDied","Data":"a9df4f962f896e3d9364613255710c5e38f442b4d3d86d54c35c9bcdfb2ede24"} Jun 06 22:17:43.872620 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:43.872585 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" event={"ID":"20a67deb-0691-4511-b6cb-dab9222d48b0","Type":"ContainerStarted","Data":"a8edb946ae51b513877fbe9c59cbbc3a0ba7351e18506566010e43cd29bf0dde"} Jun 06 22:17:43.890528 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:43.890472 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" podStartSLOduration=1.446735652 podStartE2EDuration="2.890454881s" totalImagesPullingTime="205.917337ms" totalInitContainerRuntime="1.237801892s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:17:43.889039492 +0000 UTC m=+883.637604224" watchObservedRunningTime="2026-06-06 22:17:43.890454881 +0000 UTC m=+883.639019601" Jun 06 22:17:51.406784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:51.406697 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:51.406784 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:51.406754 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:51.419058 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:51.419032 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:17:51.938683 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:17:51.938649 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-njmlp" Jun 06 22:18:00.677160 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:00.677129 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:18:00.678085 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:00.678063 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:18:32.601656 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.601623 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-999b5d6f5-x2pg8"] Jun 06 22:18:32.605154 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.605135 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.614726 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.614705 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-999b5d6f5-x2pg8"] Jun 06 22:18:32.644944 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.644919 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cxlx\" (UniqueName: \"kubernetes.io/projected/a51c2cae-6ac8-42be-87c4-ee564445aba2-kube-api-access-9cxlx\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.645052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.644953 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a51c2cae-6ac8-42be-87c4-ee564445aba2-oidc-ca\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.645052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.644974 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a51c2cae-6ac8-42be-87c4-ee564445aba2-tls-cert\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.746300 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.746272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cxlx\" (UniqueName: \"kubernetes.io/projected/a51c2cae-6ac8-42be-87c4-ee564445aba2-kube-api-access-9cxlx\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.746421 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.746305 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a51c2cae-6ac8-42be-87c4-ee564445aba2-oidc-ca\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.746421 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.746326 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a51c2cae-6ac8-42be-87c4-ee564445aba2-tls-cert\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.746925 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.746903 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a51c2cae-6ac8-42be-87c4-ee564445aba2-oidc-ca\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.748870 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.748846 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a51c2cae-6ac8-42be-87c4-ee564445aba2-tls-cert\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.764872 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.764842 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cxlx\" (UniqueName: \"kubernetes.io/projected/a51c2cae-6ac8-42be-87c4-ee564445aba2-kube-api-access-9cxlx\") pod \"authorino-999b5d6f5-x2pg8\" (UID: \"a51c2cae-6ac8-42be-87c4-ee564445aba2\") " pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:32.915457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:32.915363 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-999b5d6f5-x2pg8" Jun 06 22:18:33.046056 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:33.046024 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-999b5d6f5-x2pg8"] Jun 06 22:18:33.047283 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:18:33.047247 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda51c2cae_6ac8_42be_87c4_ee564445aba2.slice/crio-34be42c59fc830dc30c427ca63bb0113091f70f9530e0e8d47bfd4f6842c7dab WatchSource:0}: Error finding container 34be42c59fc830dc30c427ca63bb0113091f70f9530e0e8d47bfd4f6842c7dab: Status 404 returned error can't find the container with id 34be42c59fc830dc30c427ca63bb0113091f70f9530e0e8d47bfd4f6842c7dab Jun 06 22:18:33.164301 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:33.164269 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-999b5d6f5-x2pg8" event={"ID":"a51c2cae-6ac8-42be-87c4-ee564445aba2","Type":"ContainerStarted","Data":"34be42c59fc830dc30c427ca63bb0113091f70f9530e0e8d47bfd4f6842c7dab"} Jun 06 22:18:34.176655 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.176593 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-999b5d6f5-x2pg8" event={"ID":"a51c2cae-6ac8-42be-87c4-ee564445aba2","Type":"ContainerStarted","Data":"6642c6a5d1a07a5a2e12e5f323ab943cb802080ae03ca1ecdcea89f7294453f5"} Jun 06 22:18:34.203636 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.203583 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-999b5d6f5-x2pg8" podStartSLOduration=1.7859311070000001 podStartE2EDuration="2.203567644s" totalImagesPullingTime="417.636537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:18:34.198816137 +0000 UTC m=+933.947380857" watchObservedRunningTime="2026-06-06 22:18:34.203567644 +0000 UTC m=+933.952132363" Jun 06 22:18:34.235214 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.235181 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b64bbf866-xqvfg"] Jun 06 22:18:34.235441 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.235414 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" podUID="23450e96-beda-4410-8d5f-06e9ebd88a3b" containerName="authorino" containerID="cri-o://20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf" gracePeriod=30 Jun 06 22:18:34.507537 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.507514 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:18:34.563626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.563596 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/23450e96-beda-4410-8d5f-06e9ebd88a3b-oidc-ca\") pod \"23450e96-beda-4410-8d5f-06e9ebd88a3b\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " Jun 06 22:18:34.563851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.563661 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckn86\" (UniqueName: \"kubernetes.io/projected/23450e96-beda-4410-8d5f-06e9ebd88a3b-kube-api-access-ckn86\") pod \"23450e96-beda-4410-8d5f-06e9ebd88a3b\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " Jun 06 22:18:34.563851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.563697 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23450e96-beda-4410-8d5f-06e9ebd88a3b-tls-cert\") pod \"23450e96-beda-4410-8d5f-06e9ebd88a3b\" (UID: \"23450e96-beda-4410-8d5f-06e9ebd88a3b\") " Jun 06 22:18:34.565866 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.565828 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23450e96-beda-4410-8d5f-06e9ebd88a3b-kube-api-access-ckn86" (OuterVolumeSpecName: "kube-api-access-ckn86") pod "23450e96-beda-4410-8d5f-06e9ebd88a3b" (UID: "23450e96-beda-4410-8d5f-06e9ebd88a3b"). InnerVolumeSpecName "kube-api-access-ckn86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:18:34.568689 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.568660 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23450e96-beda-4410-8d5f-06e9ebd88a3b-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "23450e96-beda-4410-8d5f-06e9ebd88a3b" (UID: "23450e96-beda-4410-8d5f-06e9ebd88a3b"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:18:34.573827 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.573804 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23450e96-beda-4410-8d5f-06e9ebd88a3b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "23450e96-beda-4410-8d5f-06e9ebd88a3b" (UID: "23450e96-beda-4410-8d5f-06e9ebd88a3b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:18:34.665257 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.665228 2559 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/23450e96-beda-4410-8d5f-06e9ebd88a3b-oidc-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:18:34.665364 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.665278 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckn86\" (UniqueName: \"kubernetes.io/projected/23450e96-beda-4410-8d5f-06e9ebd88a3b-kube-api-access-ckn86\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:18:34.665364 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:34.665306 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23450e96-beda-4410-8d5f-06e9ebd88a3b-tls-cert\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" Jun 06 22:18:35.184230 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.184146 2559 generic.go:356] "Generic (PLEG): container finished" podID="23450e96-beda-4410-8d5f-06e9ebd88a3b" containerID="20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf" exitCode=0 Jun 06 22:18:35.184230 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.184193 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" Jun 06 22:18:35.184727 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.184232 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" event={"ID":"23450e96-beda-4410-8d5f-06e9ebd88a3b","Type":"ContainerDied","Data":"20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf"} Jun 06 22:18:35.184727 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.184277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b64bbf866-xqvfg" event={"ID":"23450e96-beda-4410-8d5f-06e9ebd88a3b","Type":"ContainerDied","Data":"a21267e1a68ead20d83d40d1e6a3c42f5f1d3c3967b7c9319c1203e78613e313"} Jun 06 22:18:35.184727 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.184296 2559 scope.go:117] "RemoveContainer" containerID="20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf" Jun 06 22:18:35.195646 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.195625 2559 scope.go:117] "RemoveContainer" containerID="20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf" Jun 06 22:18:35.195955 ip-10-0-136-77 kubenswrapper[2559]: E0606 22:18:35.195933 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf\": container with ID starting with 20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf not found: ID does not exist" containerID="20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf" Jun 06 22:18:35.196046 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.195960 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf"} err="failed to get container status \"20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf\": rpc error: code = NotFound desc = could not find container \"20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf\": container with ID starting with 20fe1c02f8bf0e35baf84806e1e0550d5fd70731c773f81477858c07d0fb69bf not found: ID does not exist" Jun 06 22:18:35.204008 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.203984 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b64bbf866-xqvfg"] Jun 06 22:18:35.209419 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:35.209388 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b64bbf866-xqvfg"] Jun 06 22:18:36.650807 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:36.650772 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23450e96-beda-4410-8d5f-06e9ebd88a3b" path="/var/lib/kubelet/pods/23450e96-beda-4410-8d5f-06e9ebd88a3b/volumes" Jun 06 22:18:53.859074 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:53.859038 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:18:57.756741 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:18:57.756708 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:19:05.254028 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:19:05.253993 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:19:15.359657 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:19:15.359624 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:19:24.347697 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:19:24.347614 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:19:34.359317 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:19:34.359281 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:19:43.553999 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:19:43.553964 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:19:53.545459 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:19:53.545425 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:20:26.349292 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:20:26.349255 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:20:29.547467 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:20:29.547435 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:20:34.749995 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:20:34.749964 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:20:37.544280 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:20:37.544247 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:21:05.945113 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:21:05.945023 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:21:22.541161 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:21:22.541121 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:22:00.456503 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:22:00.456469 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:22:17.556138 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:22:17.556085 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:22:30.651561 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:22:30.651529 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:22:34.248634 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:22:34.248597 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:22:47.553305 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:22:47.553270 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:22:50.552271 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:22:50.552233 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:23:00.718644 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:23:00.718617 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:23:00.720154 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:23:00.720131 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:23:25.351553 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:23:25.351513 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:23:28.351972 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:23:28.351926 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:23:50.450721 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:23:50.450633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:23:59.856795 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:23:59.856754 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:24:16.247512 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:24:16.247474 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:24:24.546069 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:24:24.546032 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:24:41.857851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:24:41.857815 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:24:50.248044 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:24:50.248009 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:25:22.652052 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:25:22.651968 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:25:31.553508 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:25:31.553470 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:25:34.447224 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:25:34.447188 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:25:40.346127 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:25:40.346080 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:25:47.855189 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:25:47.855143 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:25:56.257320 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:25:56.257280 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:26:04.552107 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:26:04.552069 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:26:07.345401 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:26:07.345368 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:26:13.256589 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:26:13.256553 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:26:25.246896 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:26:25.246864 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:01.755168 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:01.755072 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:04.053395 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:04.053358 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:11.956125 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:11.956067 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:15.948113 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:15.948072 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:23.561828 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:23.561784 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:31.050707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:31.050664 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:41.050545 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:41.050507 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:48.652670 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:48.652633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:27:57.548980 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:27:57.548942 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:00.759483 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:00.759449 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:28:00.762592 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:00.762565 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:28:01.147679 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:01.147593 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:06.347496 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:06.347457 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:15.957303 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:15.957261 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:23.877411 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:23.877323 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:33.255794 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:33.255755 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:41.158035 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:41.157999 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:50.758351 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:50.758312 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:28:59.059232 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:28:59.059194 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:29:07.951073 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:29:07.951033 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:29:15.551351 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:29:15.551306 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:29:25.774913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:29:25.774881 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:29:33.349970 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:29:33.349936 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:31:50.460689 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:31:50.460652 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:31:55.659419 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:31:55.659379 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:32:21.059827 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:32:21.059777 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:32:25.765634 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:32:25.765590 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:32:35.059120 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:32:35.059073 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:32:45.060373 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:32:45.060336 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:32:54.152031 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:32:54.151950 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:00.802188 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:00.802157 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:33:00.807565 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:00.807545 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:33:05.760131 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:05.760084 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:13.854994 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:13.854948 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:24.257005 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:24.256972 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:32.960636 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:32.960602 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:43.453031 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:43.452998 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:52.860135 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:52.860085 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:33:56.651129 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:33:56.651081 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:34:25.554807 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:34:25.554732 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:35:08.153186 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:35:08.153147 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:35:16.760805 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:35:16.760771 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:35:25.514118 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:35:25.514065 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:35:34.459087 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:35:34.459049 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:35:42.466144 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:35:42.466107 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:35:53.052742 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:35:53.052659 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:36:02.533458 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:36:02.533423 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:36:08.065447 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:36:08.065413 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:36:16.265143 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:36:16.265084 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:36:24.563837 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:36:24.563804 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:36:32.855922 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:36:32.855889 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:36:43.253778 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:36:43.253744 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:37:01.053827 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:37:01.053788 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:37:09.358042 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:37:09.358002 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:37:18.154092 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:37:18.154053 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:37:26.955392 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:37:26.955357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:37:43.658274 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:37:43.658238 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:37:51.856378 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:37:51.856335 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:00.846727 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:00.846695 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:38:00.869675 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:00.869645 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dxdx6_e2fd752e-f7f8-44e5-beaa-5aed253e7e19/ovn-acl-logging/0.log" Jun 06 22:38:00.871425 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:00.871392 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:09.366814 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:09.366769 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:17.864756 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:17.864711 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:26.953046 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:26.953011 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:35.754348 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:35.754310 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:46.662329 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:46.662298 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:38:55.755353 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:38:55.755313 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:39:06.959756 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:39:06.959721 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:39:16.052486 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:39:16.052452 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:39:23.463390 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:39:23.463355 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:39:32.162968 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:39:32.162922 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:39:38.164350 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:39:38.164313 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:39:54.254586 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:39:54.254548 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:40:03.053106 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:40:03.053072 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:40:12.154533 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:40:12.154496 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:40:18.155195 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:40:18.155159 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:40:41.658192 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:40:41.658154 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:40:53.663669 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:40:53.663629 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:40:57.252504 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:40:57.252463 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8vkk8"] Jun 06 22:41:26.601157 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:26.601122 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-999b5d6f5-x2pg8_a51c2cae-6ac8-42be-87c4-ee564445aba2/authorino/0.log" Jun 06 22:41:31.794343 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:31.794313 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-4vz5j_add74988-66e5-4395-a211-59d463702ee5/manager/0.log" Jun 06 22:41:32.135451 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:32.135373 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-qs62j_18f5a6d4-279d-49c5-a4bc-bb39413ca063/copy-manifests/0.log" Jun 06 22:41:32.141786 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:32.141763 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-qs62j_18f5a6d4-279d-49c5-a4bc-bb39413ca063/manager/0.log" Jun 06 22:41:32.983217 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:32.983188 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q_4c3a708e-c565-45bb-a322-dc7797186048/extract/0.log" Jun 06 22:41:32.988306 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:32.988279 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q_4c3a708e-c565-45bb-a322-dc7797186048/util/0.log" Jun 06 22:41:32.993379 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:32.993358 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q_4c3a708e-c565-45bb-a322-dc7797186048/pull/0.log" Jun 06 22:41:33.094631 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.094606 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv_8f20015e-cd34-47d0-8330-de59913eed85/util/0.log" Jun 06 22:41:33.100889 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.100864 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv_8f20015e-cd34-47d0-8330-de59913eed85/pull/0.log" Jun 06 22:41:33.106924 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.106897 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv_8f20015e-cd34-47d0-8330-de59913eed85/extract/0.log" Jun 06 22:41:33.212208 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.212180 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr_dfc51113-5723-4d8c-80f4-df8b6b7bb76f/util/0.log" Jun 06 22:41:33.218073 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.218056 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr_dfc51113-5723-4d8c-80f4-df8b6b7bb76f/pull/0.log" Jun 06 22:41:33.223642 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.223622 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr_dfc51113-5723-4d8c-80f4-df8b6b7bb76f/extract/0.log" Jun 06 22:41:33.330485 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.330405 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65_d14f54af-3e4e-4f19-bce8-fea912d2f999/util/0.log" Jun 06 22:41:33.336970 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.336952 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65_d14f54af-3e4e-4f19-bce8-fea912d2f999/pull/0.log" Jun 06 22:41:33.343180 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.343155 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65_d14f54af-3e4e-4f19-bce8-fea912d2f999/extract/0.log" Jun 06 22:41:33.455121 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.455070 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-999b5d6f5-x2pg8_a51c2cae-6ac8-42be-87c4-ee564445aba2/authorino/0.log" Jun 06 22:41:33.674473 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.674399 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sxtvj_1a7ccc57-fad0-4d9c-b081-17a1483c15ad/manager/0.log" Jun 06 22:41:33.779648 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:33.779622 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vk2gh_1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c/kuadrant-console-plugin/0.log" Jun 06 22:41:34.101488 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:34.101456 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8vkk8_bb69bb63-b06e-46ab-9dee-e94123fd109d/limitador/0.log" Jun 06 22:41:34.781042 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:34.781008 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76d78c7b69-8x6nz_75357243-0533-4e2d-82d2-33e5da952d5b/kube-auth-proxy/0.log" Jun 06 22:41:35.773551 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:35.773517 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-22z9h_fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482/storage-initializer/0.log" Jun 06 22:41:35.780623 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:35.780598 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-22z9h_fd1ed1e3-6e4f-4b9f-8e39-8d24930cd482/main/0.log" Jun 06 22:41:36.225578 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:36.225550 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-njmlp_20a67deb-0691-4511-b6cb-dab9222d48b0/storage-initializer/0.log" Jun 06 22:41:36.231743 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:36.231722 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-njmlp_20a67deb-0691-4511-b6cb-dab9222d48b0/main/0.log" Jun 06 22:41:40.299868 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.299826 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pkwkm/must-gather-9fpnn"] Jun 06 22:41:40.300605 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.300581 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23450e96-beda-4410-8d5f-06e9ebd88a3b" containerName="authorino" Jun 06 22:41:40.300700 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.300608 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23450e96-beda-4410-8d5f-06e9ebd88a3b" containerName="authorino" Jun 06 22:41:40.300762 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.300719 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="23450e96-beda-4410-8d5f-06e9ebd88a3b" containerName="authorino" Jun 06 22:41:40.304063 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.304042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.306545 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.306516 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pkwkm\"/\"openshift-service-ca.crt\"" Jun 06 22:41:40.306545 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.306532 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pkwkm\"/\"default-dockercfg-hl46c\"" Jun 06 22:41:40.307795 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.307773 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pkwkm\"/\"kube-root-ca.crt\"" Jun 06 22:41:40.311564 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.311276 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pkwkm/must-gather-9fpnn"] Jun 06 22:41:40.415057 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.415015 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f88b7b89-4f70-4951-9c75-84871f4e1a1d-must-gather-output\") pod \"must-gather-9fpnn\" (UID: \"f88b7b89-4f70-4951-9c75-84871f4e1a1d\") " pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.415057 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.415063 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prmbf\" (UniqueName: \"kubernetes.io/projected/f88b7b89-4f70-4951-9c75-84871f4e1a1d-kube-api-access-prmbf\") pod \"must-gather-9fpnn\" (UID: \"f88b7b89-4f70-4951-9c75-84871f4e1a1d\") " pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.516036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.515983 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f88b7b89-4f70-4951-9c75-84871f4e1a1d-must-gather-output\") pod \"must-gather-9fpnn\" (UID: \"f88b7b89-4f70-4951-9c75-84871f4e1a1d\") " pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.516036 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.516045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prmbf\" (UniqueName: \"kubernetes.io/projected/f88b7b89-4f70-4951-9c75-84871f4e1a1d-kube-api-access-prmbf\") pod \"must-gather-9fpnn\" (UID: \"f88b7b89-4f70-4951-9c75-84871f4e1a1d\") " pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.516368 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.516349 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f88b7b89-4f70-4951-9c75-84871f4e1a1d-must-gather-output\") pod \"must-gather-9fpnn\" (UID: \"f88b7b89-4f70-4951-9c75-84871f4e1a1d\") " pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.529718 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.529690 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prmbf\" (UniqueName: \"kubernetes.io/projected/f88b7b89-4f70-4951-9c75-84871f4e1a1d-kube-api-access-prmbf\") pod \"must-gather-9fpnn\" (UID: \"f88b7b89-4f70-4951-9c75-84871f4e1a1d\") " pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.614043 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.613963 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pkwkm/must-gather-9fpnn" Jun 06 22:41:40.754206 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.754178 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pkwkm/must-gather-9fpnn"] Jun 06 22:41:40.755482 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:41:40.755458 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf88b7b89_4f70_4951_9c75_84871f4e1a1d.slice/crio-4c2e57ae2b8d482e689a7818be0909491417a05a306656b8a9ff5e884600b8c9 WatchSource:0}: Error finding container 4c2e57ae2b8d482e689a7818be0909491417a05a306656b8a9ff5e884600b8c9: Status 404 returned error can't find the container with id 4c2e57ae2b8d482e689a7818be0909491417a05a306656b8a9ff5e884600b8c9 Jun 06 22:41:40.757776 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:40.757760 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:41:41.038182 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:41.038146 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/must-gather-9fpnn" event={"ID":"f88b7b89-4f70-4951-9c75-84871f4e1a1d","Type":"ContainerStarted","Data":"4c2e57ae2b8d482e689a7818be0909491417a05a306656b8a9ff5e884600b8c9"} Jun 06 22:41:42.049035 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:42.048210 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/must-gather-9fpnn" event={"ID":"f88b7b89-4f70-4951-9c75-84871f4e1a1d","Type":"ContainerStarted","Data":"0c5f458792592afa6c291756eb7b073b0d237fe2d1b8d5586c096e468acf8e04"} Jun 06 22:41:42.049035 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:42.048263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/must-gather-9fpnn" event={"ID":"f88b7b89-4f70-4951-9c75-84871f4e1a1d","Type":"ContainerStarted","Data":"df27f1eb0960e652acc86f9e86e13abffa4c06346a4877548ec8bc46a14f1da9"} Jun 06 22:41:42.066871 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:42.066814 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pkwkm/must-gather-9fpnn" podStartSLOduration=1.147194721 podStartE2EDuration="2.066798252s" totalImagesPullingTime="919.603531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:41:42.063243196 +0000 UTC m=+2321.811807916" watchObservedRunningTime="2026-06-06 22:41:42.066798252 +0000 UTC m=+2321.815362970" Jun 06 22:41:43.464910 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:43.464876 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k6f4v_31c22d88-e0a9-418a-8b27-b5fe5f2d1ef7/global-pull-secret-syncer/0.log" Jun 06 22:41:43.588813 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:43.588774 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dqkkb_c4b518bb-9e63-4eb4-b04b-4a045d66dbdf/konnectivity-agent/0.log" Jun 06 22:41:43.650000 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:43.649971 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-77.ec2.internal_6be6f86c99714a653cd42b51cc33e8d9/haproxy/0.log" Jun 06 22:41:48.306632 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.306574 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q_4c3a708e-c565-45bb-a322-dc7797186048/extract/0.log" Jun 06 22:41:48.328950 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.328920 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q_4c3a708e-c565-45bb-a322-dc7797186048/util/0.log" Jun 06 22:41:48.352389 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.352314 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597qr2q_4c3a708e-c565-45bb-a322-dc7797186048/pull/0.log" Jun 06 22:41:48.379626 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.379595 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv_8f20015e-cd34-47d0-8330-de59913eed85/extract/0.log" Jun 06 22:41:48.400995 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.400972 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv_8f20015e-cd34-47d0-8330-de59913eed85/util/0.log" Jun 06 22:41:48.424061 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.424008 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lmbqv_8f20015e-cd34-47d0-8330-de59913eed85/pull/0.log" Jun 06 22:41:48.457004 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.456914 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr_dfc51113-5723-4d8c-80f4-df8b6b7bb76f/extract/0.log" Jun 06 22:41:48.478532 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.478446 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr_dfc51113-5723-4d8c-80f4-df8b6b7bb76f/util/0.log" Jun 06 22:41:48.499192 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.499166 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m29hr_dfc51113-5723-4d8c-80f4-df8b6b7bb76f/pull/0.log" Jun 06 22:41:48.540509 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.540474 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65_d14f54af-3e4e-4f19-bce8-fea912d2f999/extract/0.log" Jun 06 22:41:48.559422 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.559345 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65_d14f54af-3e4e-4f19-bce8-fea912d2f999/util/0.log" Jun 06 22:41:48.583306 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.583277 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16qr65_d14f54af-3e4e-4f19-bce8-fea912d2f999/pull/0.log" Jun 06 22:41:48.839618 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.839530 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-999b5d6f5-x2pg8_a51c2cae-6ac8-42be-87c4-ee564445aba2/authorino/0.log" Jun 06 22:41:48.900533 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.900502 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sxtvj_1a7ccc57-fad0-4d9c-b081-17a1483c15ad/manager/0.log" Jun 06 22:41:48.927777 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:48.927751 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vk2gh_1d1f8e1d-e46b-45bd-ac7a-62cbec57f05c/kuadrant-console-plugin/0.log" Jun 06 22:41:49.078200 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:49.078163 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8vkk8_bb69bb63-b06e-46ab-9dee-e94123fd109d/limitador/0.log" Jun 06 22:41:50.875408 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:50.875374 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/alertmanager/0.log" Jun 06 22:41:50.897457 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:50.897428 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/config-reloader/0.log" Jun 06 22:41:50.920081 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:50.920022 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/kube-rbac-proxy-web/0.log" Jun 06 22:41:50.942745 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:50.942716 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/kube-rbac-proxy/0.log" Jun 06 22:41:50.967064 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:50.967003 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/kube-rbac-proxy-metric/0.log" Jun 06 22:41:50.990713 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:50.990685 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/prom-label-proxy/0.log" Jun 06 22:41:51.012668 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.012622 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_223bbe4e-2494-4e28-983e-31ae72b314b5/init-config-reloader/0.log" Jun 06 22:41:51.053498 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.053399 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-kcnzv_e8f6cde2-e582-46f0-9d75-bbe9327855cb/cluster-monitoring-operator/0.log" Jun 06 22:41:51.375115 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.375060 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk9wz_ab774aa9-ca08-4ed7-8599-53a27ef26305/node-exporter/0.log" Jun 06 22:41:51.397279 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.397246 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk9wz_ab774aa9-ca08-4ed7-8599-53a27ef26305/kube-rbac-proxy/0.log" Jun 06 22:41:51.420461 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.420437 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk9wz_ab774aa9-ca08-4ed7-8599-53a27ef26305/init-textfile/0.log" Jun 06 22:41:51.460136 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.460038 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-tzbqc_170582cb-e5e7-4259-9366-c50f727de250/kube-rbac-proxy-main/0.log" Jun 06 22:41:51.486861 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.486811 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-tzbqc_170582cb-e5e7-4259-9366-c50f727de250/kube-rbac-proxy-self/0.log" Jun 06 22:41:51.522677 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.522647 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-tzbqc_170582cb-e5e7-4259-9366-c50f727de250/openshift-state-metrics/0.log" Jun 06 22:41:51.922486 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.922458 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/thanos-query/0.log" Jun 06 22:41:51.960251 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.960147 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy-web/0.log" Jun 06 22:41:51.990361 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:51.990313 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy/0.log" Jun 06 22:41:52.042478 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:52.042405 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/prom-label-proxy/0.log" Jun 06 22:41:52.066793 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:52.066750 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy-rules/0.log" Jun 06 22:41:52.093278 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:52.093245 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b46454b-kgf5p_20d62d13-296a-4028-a99c-92007545fa1c/kube-rbac-proxy-metrics/0.log" Jun 06 22:41:54.875549 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:54.875517 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6cb49bf655-qqznd_4fc9f7ed-53b9-4ae9-b9b5-4d45f7363adf/console/0.log" Jun 06 22:41:54.907079 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:54.907048 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-d772h_9391c549-e6c6-4f24-ad76-7dd728cc585f/download-server/0.log" Jun 06 22:41:56.027931 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.027887 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f"] Jun 06 22:41:56.038071 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.038039 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.039112 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.039061 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f"] Jun 06 22:41:56.201947 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.201898 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-proc\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.202162 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.201956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22t2h\" (UniqueName: \"kubernetes.io/projected/73417df6-e2a2-4bee-b480-4601ef8dc302-kube-api-access-22t2h\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.202162 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.201989 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-podres\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.202162 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.202134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-sys\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.202365 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.202170 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-lib-modules\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303449 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303350 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-sys\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303449 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-lib-modules\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303519 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-sys\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-proc\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303604 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22t2h\" (UniqueName: \"kubernetes.io/projected/73417df6-e2a2-4bee-b480-4601ef8dc302-kube-api-access-22t2h\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303621 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-lib-modules\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-podres\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.303707 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303688 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-proc\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.304024 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.303755 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/73417df6-e2a2-4bee-b480-4601ef8dc302-podres\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.312255 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.312229 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22t2h\" (UniqueName: \"kubernetes.io/projected/73417df6-e2a2-4bee-b480-4601ef8dc302-kube-api-access-22t2h\") pod \"perf-node-gather-daemonset-4v66f\" (UID: \"73417df6-e2a2-4bee-b480-4601ef8dc302\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.351699 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.351662 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:56.540546 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.540512 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f"] Jun 06 22:41:56.541661 ip-10-0-136-77 kubenswrapper[2559]: W0606 22:41:56.541622 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod73417df6_e2a2_4bee_b480_4601ef8dc302.slice/crio-62aee13436e251a3dd00b994db07606be80fa65b2fb2a6201f513ab0869aa0b3 WatchSource:0}: Error finding container 62aee13436e251a3dd00b994db07606be80fa65b2fb2a6201f513ab0869aa0b3: Status 404 returned error can't find the container with id 62aee13436e251a3dd00b994db07606be80fa65b2fb2a6201f513ab0869aa0b3 Jun 06 22:41:56.881307 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.881231 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tmt8x_78c6ae3d-645e-4097-b9a8-68426888d9f2/dns/0.log" Jun 06 22:41:56.901989 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.901964 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tmt8x_78c6ae3d-645e-4097-b9a8-68426888d9f2/kube-rbac-proxy/0.log" Jun 06 22:41:56.924080 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:56.924048 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5v4lt_bb57a9c4-fe3e-4f7f-b452-011eee4d5173/dns-node-resolver/0.log" Jun 06 22:41:57.179544 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:57.179454 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" event={"ID":"73417df6-e2a2-4bee-b480-4601ef8dc302","Type":"ContainerStarted","Data":"8a2cc6d549926385b0ac313de92791a906e2fca1d4df735d7c87d40f3b8683a4"} Jun 06 22:41:57.179544 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:57.179521 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" event={"ID":"73417df6-e2a2-4bee-b480-4601ef8dc302","Type":"ContainerStarted","Data":"62aee13436e251a3dd00b994db07606be80fa65b2fb2a6201f513ab0869aa0b3"} Jun 06 22:41:57.180020 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:57.179642 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:41:57.507270 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:57.507235 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9b55d5d4c-fghf8_aeb90dca-a413-4e71-a86c-49236305ea78/registry/0.log" Jun 06 22:41:57.519474 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:57.519442 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9b55d5d4c-fghf8_aeb90dca-a413-4e71-a86c-49236305ea78/registry/1.log" Jun 06 22:41:57.589245 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:57.589218 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xh4kn_c365b91d-ec96-452a-a635-fffdaf8f6f2c/node-ca/0.log" Jun 06 22:41:58.671949 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:58.671924 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76d78c7b69-8x6nz_75357243-0533-4e2d-82d2-33e5da952d5b/kube-auth-proxy/0.log" Jun 06 22:41:59.553880 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:41:59.553843 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rc8nj_78fb49bc-b80f-4ade-85be-ae550676a352/serve-healthcheck-canary/0.log" Jun 06 22:42:00.051919 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:00.051883 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-hpmgc_e36d27da-a653-45a8-b223-a2f2118773ca/insights-operator/0.log" Jun 06 22:42:00.054675 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:00.054646 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-hpmgc_e36d27da-a653-45a8-b223-a2f2118773ca/insights-operator/1.log" Jun 06 22:42:00.326913 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:00.326842 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qknqk_5b0c63a6-59a9-45ff-9f88-a46568a5db8f/kube-rbac-proxy/0.log" Jun 06 22:42:00.365662 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:00.365631 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qknqk_5b0c63a6-59a9-45ff-9f88-a46568a5db8f/exporter/0.log" Jun 06 22:42:00.403362 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:00.403333 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qknqk_5b0c63a6-59a9-45ff-9f88-a46568a5db8f/extractor/0.log" Jun 06 22:42:03.053851 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:03.053817 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-4vz5j_add74988-66e5-4395-a211-59d463702ee5/manager/0.log" Jun 06 22:42:03.198656 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:03.198620 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" Jun 06 22:42:03.213563 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:03.213495 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-4v66f" podStartSLOduration=7.213456514 podStartE2EDuration="7.213456514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:41:57.1977524 +0000 UTC m=+2336.946317123" watchObservedRunningTime="2026-06-06 22:42:03.213456514 +0000 UTC m=+2342.962021231" Jun 06 22:42:03.302985 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:03.302956 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-qs62j_18f5a6d4-279d-49c5-a4bc-bb39413ca063/manager/0.log" Jun 06 22:42:03.325158 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:03.325054 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-qs62j_18f5a6d4-279d-49c5-a4bc-bb39413ca063/copy-manifests/0.log" Jun 06 22:42:04.937606 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:04.937518 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7774b9f8d5-m8h67_50d6849e-44bc-4907-aece-641d47ec47b9/manager/0.log" Jun 06 22:42:09.406078 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:09.406040 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76d78c7b69-8x6nz_75357243-0533-4e2d-82d2-33e5da952d5b/kube-auth-proxy/0.log" Jun 06 22:42:10.302176 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:10.302149 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-nfckm_b4429180-f6f8-4e31-bb06-d953bbe832fe/migrator/0.log" Jun 06 22:42:10.324715 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:10.324690 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-nfckm_b4429180-f6f8-4e31-bb06-d953bbe832fe/graceful-termination/0.log" Jun 06 22:42:10.826830 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:10.826795 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76d78c7b69-8x6nz_75357243-0533-4e2d-82d2-33e5da952d5b/kube-auth-proxy/0.log" Jun 06 22:42:12.266847 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.266822 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/kube-multus-additional-cni-plugins/0.log" Jun 06 22:42:12.288392 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.288363 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/egress-router-binary-copy/0.log" Jun 06 22:42:12.309191 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.309163 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/cni-plugins/0.log" Jun 06 22:42:12.331807 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.331781 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/bond-cni-plugin/0.log" Jun 06 22:42:12.352639 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.352561 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/routeoverride-cni/0.log" Jun 06 22:42:12.373296 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.373270 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/whereabouts-cni-bincopy/0.log" Jun 06 22:42:12.393732 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.393703 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qjk7t_b24c7544-f928-4f19-ba03-5f51f474f7f8/whereabouts-cni/0.log" Jun 06 22:42:12.429394 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.429368 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-p6gg8_e51cf442-9dc6-436e-93f3-aaf840764973/kube-multus/0.log" Jun 06 22:42:12.509333 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.509307 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-978qx_7635dc64-f99d-4422-99ff-419efdcefc01/network-metrics-daemon/0.log" Jun 06 22:42:12.528174 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.528113 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-978qx_7635dc64-f99d-4422-99ff-419efdcefc01/kube-rbac-proxy/0.log" Jun 06 22:42:12.664755 ip-10-0-136-77 kubenswrapper[2559]: I0606 22:42:12.664676 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76d78c7b69-8x6nz_75357243-0533-4e2d-82d2-33e5da952d5b/kube-auth-proxy/0.log"