Jul 01 23:04:28.244578 ip-10-0-141-234 systemd[1]: Starting Kubernetes Kubelet... Jul 01 23:04:30.100860 ip-10-0-141-234 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 23:04:30.100860 ip-10-0-141-234 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 23:04:30.100860 ip-10-0-141-234 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 23:04:30.100860 ip-10-0-141-234 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 23:04:30.100860 ip-10-0-141-234 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 23:04:30.103926 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.103278 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106851 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106862 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106867 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106870 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106873 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106875 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106878 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106884 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106887 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106889 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106892 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106894 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106897 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106900 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106902 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106905 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106907 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106910 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106913 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 23:04:30.168348 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106915 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106918 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106920 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106922 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106925 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106929 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106934 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106937 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106940 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106943 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106946 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106948 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106951 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106953 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106957 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106961 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106964 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106967 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106970 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 23:04:30.169953 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106972 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106975 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106978 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106981 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106983 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106986 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106988 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106991 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106993 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106996 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.106998 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107001 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107004 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107006 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107009 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107012 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107014 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107016 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107019 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107022 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 23:04:30.170624 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107026 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107031 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107034 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107037 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107040 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107043 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107045 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107048 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107050 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107053 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107056 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107059 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107061 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107064 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107067 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107069 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107072 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107075 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107077 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107080 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 23:04:30.171897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107083 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107085 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107088 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107090 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107093 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107095 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107098 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107100 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107103 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107105 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107108 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107110 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107113 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107117 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107120 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107122 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107125 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107128 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107130 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107133 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 23:04:30.172538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.107136 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135588 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135600 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135621 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135624 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135627 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135631 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135634 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135637 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135640 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135643 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135646 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135649 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135652 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135656 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135658 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135661 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135664 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135667 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 23:04:30.173640 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135669 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135672 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135674 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135677 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135680 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135682 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135685 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135688 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135691 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135693 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135696 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135698 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135701 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135704 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135706 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135709 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135711 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135714 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135716 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 23:04:30.174310 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135719 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135721 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135723 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135726 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135728 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135731 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135733 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135736 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135738 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135741 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135743 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135746 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135749 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135751 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135754 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135757 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135759 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135762 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135764 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135767 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 23:04:30.175459 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135770 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135773 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135776 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135778 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135781 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135783 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135788 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135792 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135795 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135798 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135801 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135804 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135807 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135809 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135811 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135814 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135818 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135821 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135823 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135826 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 23:04:30.176312 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135829 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135831 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135834 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135837 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135839 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135844 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135855 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135858 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135861 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135863 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135866 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135869 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135871 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135874 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135877 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135879 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135882 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135884 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135887 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135890 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 23:04:30.177156 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135893 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.135896 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136318 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136323 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136326 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136328 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136331 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136334 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136338 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136342 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136347 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136350 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136353 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136355 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136358 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136361 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136364 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136367 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136375 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136378 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 23:04:30.180034 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136381 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136384 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136386 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136389 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136391 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136395 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136397 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136400 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136402 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136405 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136408 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136411 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136414 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136416 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136418 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136421 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136424 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136426 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136429 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136431 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 23:04:30.180857 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136434 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136436 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136439 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136442 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136444 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136447 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136449 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136451 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136454 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136456 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136459 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136467 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136470 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136474 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136476 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136479 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136482 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136484 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136487 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 23:04:30.181491 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136490 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136492 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136495 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136498 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136500 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136503 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136505 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136508 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136510 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136513 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136516 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136518 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136521 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136524 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136527 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136530 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136532 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136535 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136538 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136541 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 23:04:30.182138 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136543 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136546 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136548 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136551 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136553 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136561 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136564 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136567 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136569 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136572 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136574 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136577 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136579 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136582 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136584 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136587 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136589 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136592 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136594 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136597 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 23:04:30.182781 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136600 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.136619 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136696 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136710 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136721 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136726 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136729 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136732 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136736 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136740 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136743 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136746 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136750 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136753 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136756 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136759 2569 flags.go:64] FLAG: --cgroup-root="" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136762 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136764 2569 flags.go:64] FLAG: --client-ca-file="" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136773 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136776 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136782 2569 flags.go:64] FLAG: --cluster-domain="" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136785 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136788 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 23:04:30.183417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136792 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136795 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136799 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136802 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136805 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136808 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136811 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136813 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136816 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136819 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136822 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136826 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136829 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136832 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136835 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136838 2569 flags.go:64] FLAG: --enable-server="true" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136841 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136848 2569 flags.go:64] FLAG: --event-burst="100" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136851 2569 flags.go:64] FLAG: --event-qps="50" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136854 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136857 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136860 2569 flags.go:64] FLAG: --eviction-hard="" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136864 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136869 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136872 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 23:04:30.184049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136875 2569 flags.go:64] FLAG: --eviction-soft="" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136878 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136881 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136889 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136892 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136895 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136898 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136901 2569 flags.go:64] FLAG: --feature-gates="" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136904 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136907 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136910 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136913 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136916 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136919 2569 flags.go:64] FLAG: --help="false" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136922 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-141-234.ec2.internal" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136926 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136929 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136934 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136937 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136940 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136943 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136946 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136949 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 23:04:30.186999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136952 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136955 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136958 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136960 2569 flags.go:64] FLAG: --kube-reserved="" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136963 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136966 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136969 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136973 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136976 2569 flags.go:64] FLAG: --lock-file="" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136979 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136982 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136985 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136990 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.136998 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137001 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137004 2569 flags.go:64] FLAG: --logging-format="text" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137007 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137010 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137013 2569 flags.go:64] FLAG: --manifest-url="" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137016 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137020 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137023 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137027 2569 flags.go:64] FLAG: --max-pods="110" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137030 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137033 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 23:04:30.187883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137035 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137040 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137043 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137046 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137048 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137055 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137058 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137061 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137064 2569 flags.go:64] FLAG: --pod-cidr="" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137067 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137072 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137075 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137078 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137081 2569 flags.go:64] FLAG: --port="10250" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137084 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137088 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c55bebd4e430ea02" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137091 2569 flags.go:64] FLAG: --qos-reserved="" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137094 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137097 2569 flags.go:64] FLAG: --register-node="true" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137100 2569 flags.go:64] FLAG: --register-with-taints="" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137103 2569 flags.go:64] FLAG: --registry-burst="10" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137112 2569 flags.go:64] FLAG: --registry-qps="5" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137115 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137117 2569 flags.go:64] FLAG: --reserved-memory="" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137121 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 23:04:30.188896 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137124 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137127 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137130 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137132 2569 flags.go:64] FLAG: --runonce="false" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137135 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137138 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137141 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137144 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137148 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137151 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137154 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137157 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137160 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137162 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137165 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137168 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137171 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137174 2569 flags.go:64] FLAG: --system-cgroups="" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137177 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137182 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137185 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137188 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137195 2569 flags.go:64] FLAG: --tls-min-version="" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137198 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137201 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 23:04:30.190361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137204 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137207 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137210 2569 flags.go:64] FLAG: --v="2" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137214 2569 flags.go:64] FLAG: --version="false" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137228 2569 flags.go:64] FLAG: --vmodule="" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137231 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.137235 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137370 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137374 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137377 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137382 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137385 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137388 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137390 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137393 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137397 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137400 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137403 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137405 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137408 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137410 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 23:04:30.191107 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137413 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137415 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137418 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137421 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137423 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137426 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137428 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137431 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137435 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137437 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137440 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137442 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137445 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137448 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137450 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137453 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137462 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137465 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137467 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137470 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 23:04:30.191770 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137473 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137475 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137477 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137480 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137482 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137485 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137489 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137491 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137493 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137496 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137499 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137501 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137503 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137506 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137509 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137511 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137514 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137516 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137519 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137522 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 23:04:30.192392 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137525 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137528 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137531 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137533 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137536 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137539 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137542 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137544 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137547 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137555 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137557 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137560 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137563 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137567 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137570 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137573 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137576 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137579 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137583 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 23:04:30.193101 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137586 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137589 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137591 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137594 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137596 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137599 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137602 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137620 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137624 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137627 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137631 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137633 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137636 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137640 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137643 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137646 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137649 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137652 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137655 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 23:04:30.193738 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137657 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137660 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137663 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137665 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137674 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137676 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.137679 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.138351 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.144747 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.144762 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.144779 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.144785 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144859 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144864 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 23:04:30.196561 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144867 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144870 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144873 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144876 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144879 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144882 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144885 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144888 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144891 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144893 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144896 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144901 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144904 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144908 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144910 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144913 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144915 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144918 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144921 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144924 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 23:04:30.197666 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144927 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144929 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144933 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144935 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144938 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144940 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144943 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144945 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144948 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144951 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144953 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144956 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144960 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144962 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144965 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144967 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144970 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144973 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144975 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144978 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 23:04:30.198479 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144980 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144983 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144986 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144991 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144993 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.144997 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145000 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145003 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145006 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145008 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145011 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145013 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145016 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145018 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145021 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145024 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145027 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145029 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145032 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145034 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 23:04:30.200195 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145037 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145039 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145042 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145045 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145047 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145050 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145052 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145056 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145060 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145063 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145067 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145071 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145074 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145077 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145080 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145084 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145088 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145095 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 23:04:30.201064 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145098 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145101 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145103 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145106 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145109 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145112 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145115 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145118 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145121 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145123 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145126 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145129 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145132 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145135 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145137 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145140 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145142 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145145 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 23:04:30.202119 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145147 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.145152 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145258 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145263 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145266 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145269 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145272 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145274 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145277 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145281 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145283 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145286 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145289 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145293 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 23:04:30.202958 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145296 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145298 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145301 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145305 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145308 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145311 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145314 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145317 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145320 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145323 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145325 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145328 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145330 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145333 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145335 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145338 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145341 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145343 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145346 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145348 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 23:04:30.204019 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145351 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145353 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145356 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145359 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145361 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145364 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145366 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145369 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145372 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145374 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145378 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145382 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145384 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145387 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145389 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145392 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145394 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145397 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145400 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145402 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 23:04:30.204882 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145405 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145407 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145410 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145413 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145416 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145418 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145421 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145423 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145426 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145428 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145431 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145433 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145436 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145439 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145441 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145443 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145446 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145448 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145451 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 23:04:30.206361 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145455 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145458 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145461 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145465 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145469 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145472 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145474 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145477 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145480 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145482 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145485 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145487 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145490 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145493 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145495 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145498 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145500 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145502 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145506 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145508 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 23:04:30.207384 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145511 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145515 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145517 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145520 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145522 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145525 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145528 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:30.145530 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.145535 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.145639 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.147947 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.149393 2569 server.go:1019] "Starting client certificate rotation" Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.149487 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.149525 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 23:04:30.281287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.151917 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 23:04:30.212043 ip-10-0-141-234 systemd[1]: Started Kubernetes Kubelet. Jul 01 23:04:30.282203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.155113 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 23:04:30.282203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.169389 2569 log.go:25] "Validated CRI v1 runtime API" Jul 01 23:04:30.282203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.174991 2569 log.go:25] "Validated CRI v1 image API" Jul 01 23:04:30.282203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.175821 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 23:04:30.282203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.179730 2569 fs.go:135] Filesystem UUIDs: map[5325c330-1f84-4431-967d-d0f3d4b5a22c:/dev/nvme0n1p4 5aa9de4a-c055-4181-8271-9f20189039e4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 01 23:04:30.282203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.179754 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.185907 2569 manager.go:216] Machine: {Timestamp:2026-07-01 23:04:30.184626709 +0000 UTC m=+0.407154930 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098811 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f70baff70a5d31edb86bcea34e514 SystemUUID:ec2f70ba-ff70-a5d3-1edb-86bcea34e514 BootID:13b74241-0bdf-4603-8b29-417d979cf24e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:99:72:88:4e:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:99:72:88:4e:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:6c:45:6c:ef:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.186004 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.186077 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.189932 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.189967 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-234.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.191216 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.191236 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.191247 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.191909 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.192684 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.197252 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.197381 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198052 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198069 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198089 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198101 2569 kubelet.go:398] "Adding apiserver pod source" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198114 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 23:04:30.283410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198456 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.198476 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.201047 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.202637 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.202683 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203676 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203692 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203699 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203706 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203716 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203722 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203732 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203739 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203748 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203754 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203773 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.203780 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.204358 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.204369 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.208116 2569 server.go:1268] "Started kubelet" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.208797 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.208958 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.208986 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.212408 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.218731 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.219909 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.220439 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-234.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.220101 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.223639 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.224340 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.224363 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.224429 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.224894 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.224907 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.224970 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225020 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225029 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.222379 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-234.ec2.internal.18be4ce9f5139ce3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-234.ec2.internal,UID:ip-10-0-141-234.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-234.ec2.internal,},FirstTimestamp:2026-07-01 23:04:30.208089315 +0000 UTC m=+0.430617518,LastTimestamp:2026-07-01 23:04:30.208089315 +0000 UTC m=+0.430617518,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-234.ec2.internal,}" Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225429 2569 factory.go:153] Registering CRI-O factory Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225444 2569 factory.go:223] Registration of the crio container factory successfully Jul 01 23:04:30.284310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225496 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225505 2569 factory.go:55] Registering systemd factory Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225513 2569 factory.go:223] Registration of the systemd container factory successfully Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225540 2569 factory.go:103] Registering Raw factory Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.225554 2569 manager.go:1196] Started watching for new ooms in manager Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.226045 2569 manager.go:349] Starting recovery of all containers Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.227080 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.245990 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.256493 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.257313 2569 manager.go:354] Recovery completed Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.263314 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265246 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265269 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265279 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265744 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265754 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265771 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.265755 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.267346 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.267373 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.267386 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.267486 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-234.ec2.internal.18be4ce9f87bec9a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-234.ec2.internal,UID:ip-10-0-141-234.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-234.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-234.ec2.internal,},FirstTimestamp:2026-07-01 23:04:30.265257114 +0000 UTC m=+0.487785315,LastTimestamp:2026-07-01 23:04:30.265257114 +0000 UTC m=+0.487785315,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-234.ec2.internal,}" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.267839 2569 policy_none.go:49] "None policy: Start" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.267855 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.267869 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.268297 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-47np2" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.269904 2569 policy_none.go:47] "Start" Jul 01 23:04:30.286087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.272583 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-47np2" Jul 01 23:04:30.690162 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.687898 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:30.694784 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.693066 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-234.ec2.internal.18be4ce9f87c2efa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-234.ec2.internal,UID:ip-10-0-141-234.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-234.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-234.ec2.internal,},FirstTimestamp:2026-07-01 23:04:30.265274106 +0000 UTC m=+0.487802307,LastTimestamp:2026-07-01 23:04:30.265274106 +0000 UTC m=+0.487802307,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-234.ec2.internal,}" Jul 01 23:04:30.694784 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.693121 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 01 23:04:30.801272 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.788252 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:30.889035 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.889010 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.914934 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.916208 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.916243 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.916283 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.916314 2569 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.918683 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921152 2569 manager.go:337] "Starting Device Plugin manager" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.921179 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921191 2569 server.go:87] "Starting device plugin registration server" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921452 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921466 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921541 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921620 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:30.921634 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.922154 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 23:04:30.922944 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:30.922198 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.016640 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.016573 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal"] Jul 01 23:04:31.016812 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.016690 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:31.017729 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.017706 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:31.017838 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.017739 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:31.017838 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.017749 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:31.019763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.019751 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:31.019899 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.019885 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.019942 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.019913 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:31.020519 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.020502 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:31.020597 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.020528 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:31.020597 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.020550 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:31.020597 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.020563 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:31.020597 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.020533 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:31.020736 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.020624 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:31.022355 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.022341 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:31.022618 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.022591 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.022692 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.022641 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 23:04:31.022995 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.022978 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:31.023074 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.023007 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:31.023074 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.023018 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:31.023074 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.023035 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.023376 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.023363 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientMemory" Jul 01 23:04:31.023446 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.023393 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 23:04:31.023446 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.023409 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeHasSufficientPID" Jul 01 23:04:31.031490 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.031478 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.031547 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.031496 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-234.ec2.internal\": node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.047706 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.047687 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.055299 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.055262 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-234.ec2.internal\" not found" node="ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.059696 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.059681 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-234.ec2.internal\" not found" node="ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.089980 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.089962 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2ca051fa183ca1da2ae0341414c5a72-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal\" (UID: \"f2ca051fa183ca1da2ae0341414c5a72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.090044 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.089985 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2ca051fa183ca1da2ae0341414c5a72-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal\" (UID: \"f2ca051fa183ca1da2ae0341414c5a72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.090044 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.090004 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec0e47b4d0f2d635d3199e833873acd-config\") pod \"kube-apiserver-proxy-ip-10-0-141-234.ec2.internal\" (UID: \"cec0e47b4d0f2d635d3199e833873acd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.131015 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.130993 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 23:04:31.148236 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.148208 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.148236 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.148219 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 23:04:31.148366 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.148341 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 23:04:31.148410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.148390 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 23:04:31.148477 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.148395 2569 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a2f25b6c2002a4e5b887a1b808d44487-f2b40fc74564307c.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-141-234.ec2.internal.18be4ce9f87c2efa\": read tcp 10.0.141.234:57166->100.49.215.41:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-141-234.ec2.internal.18be4ce9f87c2efa default 6237 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-234.ec2.internal,UID:ip-10-0-141-234.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-234.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-234.ec2.internal,},FirstTimestamp:2026-07-01 23:04:30 +0000 UTC,LastTimestamp:2026-07-01 23:04:31.023012235 +0000 UTC m=+1.245540439,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-234.ec2.internal,}" Jul 01 23:04:31.190623 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.190590 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec0e47b4d0f2d635d3199e833873acd-config\") pod \"kube-apiserver-proxy-ip-10-0-141-234.ec2.internal\" (UID: \"cec0e47b4d0f2d635d3199e833873acd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.190725 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.190640 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2ca051fa183ca1da2ae0341414c5a72-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal\" (UID: \"f2ca051fa183ca1da2ae0341414c5a72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.190725 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.190660 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2ca051fa183ca1da2ae0341414c5a72-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal\" (UID: \"f2ca051fa183ca1da2ae0341414c5a72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.190725 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.190687 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec0e47b4d0f2d635d3199e833873acd-config\") pod \"kube-apiserver-proxy-ip-10-0-141-234.ec2.internal\" (UID: \"cec0e47b4d0f2d635d3199e833873acd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.190725 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.190700 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2ca051fa183ca1da2ae0341414c5a72-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal\" (UID: \"f2ca051fa183ca1da2ae0341414c5a72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.190879 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.190759 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2ca051fa183ca1da2ae0341414c5a72-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal\" (UID: \"f2ca051fa183ca1da2ae0341414c5a72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.248947 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.248925 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.349205 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.349149 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.358310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.358293 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.363153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.363134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.450269 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.450243 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.489382 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.489353 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 23:04:31.550402 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:31.550370 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-234.ec2.internal\" not found" Jul 01 23:04:31.551784 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.551769 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 23:04:31.624863 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.624779 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.637601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.637574 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 23:04:31.638687 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.638673 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" Jul 01 23:04:31.650972 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.650956 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 23:04:31.691767 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.691735 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 22:59:30 +0000 UTC" deadline="2028-02-12 16:15:38.765896782 +0000 UTC" Jul 01 23:04:31.691767 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.691767 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14177h11m7.074134058s" Jul 01 23:04:31.950377 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:31.950344 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ca051fa183ca1da2ae0341414c5a72.slice/crio-b59bc30137470850caa133a8d677b44a48c71a76907ae0376a140cca6368ad1f WatchSource:0}: Error finding container b59bc30137470850caa133a8d677b44a48c71a76907ae0376a140cca6368ad1f: Status 404 returned error can't find the container with id b59bc30137470850caa133a8d677b44a48c71a76907ae0376a140cca6368ad1f Jul 01 23:04:31.950730 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:31.950713 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec0e47b4d0f2d635d3199e833873acd.slice/crio-2b437f0fc0df88cc24ed6767a09ae3dd11130612f505c981645cae33b249497f WatchSource:0}: Error finding container 2b437f0fc0df88cc24ed6767a09ae3dd11130612f505c981645cae33b249497f: Status 404 returned error can't find the container with id 2b437f0fc0df88cc24ed6767a09ae3dd11130612f505c981645cae33b249497f Jul 01 23:04:31.956176 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:31.956158 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 23:04:32.200418 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.200359 2569 apiserver.go:52] "Watching apiserver" Jul 01 23:04:32.210723 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.210697 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 23:04:32.211069 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.211049 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal","openshift-network-diagnostics/network-check-target-4pz9m","openshift-network-operator/iptables-alerter-2bqdg","kube-system/global-pull-secret-syncer-xjqgl","kube-system/konnectivity-agent-tz5vp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg","openshift-dns/node-resolver-vb4f8","openshift-multus/multus-additional-cni-plugins-b8j64","openshift-multus/multus-xdblm","openshift-multus/network-metrics-daemon-ghqjk","openshift-ovn-kubernetes/ovnkube-node-q5hbn","kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mct7s","openshift-image-registry/node-ca-l8tqd"] Jul 01 23:04:32.214924 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.214907 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:32.215017 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.214997 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:32.217008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.216990 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.219211 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.219193 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.219369 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.219250 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:32.219522 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.219481 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.219522 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.219517 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hj49m\"" Jul 01 23:04:32.219644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.219546 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.219644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.219632 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 23:04:32.221287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.221261 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.223224 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.223210 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 23:04:32.223342 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.223323 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 23:04:32.223390 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.223329 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2564l\"" Jul 01 23:04:32.223478 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.223463 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.225189 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.225176 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 23:04:32.225508 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.225484 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 23:04:32.225585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.225560 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.226044 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.226013 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-swzt5\"" Jul 01 23:04:32.226171 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.226156 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.226288 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.226272 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.227960 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.227944 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.230166 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.230150 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 23:04:32.230302 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.230269 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.230378 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.230365 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.230453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.230421 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.231139 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.231120 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 23:04:32.231252 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.231196 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q6qm9\"" Jul 01 23:04:32.231252 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.231217 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.231426 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.231410 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 23:04:32.231499 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.231485 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gbm88\"" Jul 01 23:04:32.231764 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.231749 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.232224 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.232207 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 23:04:32.232303 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.232288 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gxclh\"" Jul 01 23:04:32.232701 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.232686 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:32.232778 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.232743 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:32.235190 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.235175 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.239262 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.237387 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 23:04:32.239770 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.239749 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 23:04:32.239872 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.239790 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 23:04:32.239872 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.239859 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 23:04:32.240047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.240027 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.240290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.240276 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.240369 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.240360 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qxkl7\"" Jul 01 23:04:32.240430 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.240370 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 23:04:32.240430 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.240370 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.242265 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.242247 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.242356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.242277 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cwmhv\"" Jul 01 23:04:32.242356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.242293 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.243472 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.243457 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.244327 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.244313 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 23:04:32.244411 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.244395 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-88hfn\"" Jul 01 23:04:32.244707 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.244687 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 23:04:32.245337 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.245319 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 23:04:32.275259 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.275246 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g78kj" Jul 01 23:04:32.284467 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.284451 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g78kj" Jul 01 23:04:32.296845 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296823 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a09e14eb-e908-4ba5-afeb-c85498121530-iptables-alerter-script\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.296919 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296847 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpscq\" (UniqueName: \"kubernetes.io/projected/a09e14eb-e908-4ba5-afeb-c85498121530-kube-api-access-zpscq\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.296919 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-device-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.296919 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296878 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.297013 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a09e14eb-e908-4ba5-afeb-c85498121530-host-slash\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.297013 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296944 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afd99605-6853-41d1-8b9f-a4aa46695472-dbus\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.297013 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296961 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.297013 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.296992 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6d6e1ef-fd1d-4a47-a28c-16b1343110b9-konnectivity-ca\") pod \"konnectivity-agent-tz5vp\" (UID: \"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9\") " pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297015 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297031 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4msqt\" (UniqueName: \"kubernetes.io/projected/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-kube-api-access-4msqt\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297044 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8n78\" (UniqueName: \"kubernetes.io/projected/3e982204-a919-4ae0-9ea7-1af013a9324d-kube-api-access-h8n78\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297060 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afd99605-6853-41d1-8b9f-a4aa46695472-kubelet-config\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297091 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6d6e1ef-fd1d-4a47-a28c-16b1343110b9-agent-certs\") pod \"konnectivity-agent-tz5vp\" (UID: \"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9\") " pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297107 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-socket-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.297127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297121 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-registration-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.297317 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297135 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e982204-a919-4ae0-9ea7-1af013a9324d-hosts-file\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.297317 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297158 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e982204-a919-4ae0-9ea7-1af013a9324d-tmp-dir\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.297317 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297179 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:32.297317 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.297199 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-sys-fs\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.325496 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.325470 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 23:04:32.325670 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.325655 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 23:04:32.398063 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398037 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovnkube-script-lib\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.398063 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-modprobe-d\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398082 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-sys\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398098 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-var-lib-kubelet\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398138 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-host\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398167 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afd99605-6853-41d1-8b9f-a4aa46695472-kubelet-config\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398184 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2rds\" (UniqueName: \"kubernetes.io/projected/06f9552e-9efb-4688-8ed1-10c3e64f4249-kube-api-access-l2rds\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398200 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-etc-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398217 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-sys-fs\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-system-cni-dir\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398246 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-cnibin\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.398264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398254 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afd99605-6853-41d1-8b9f-a4aa46695472-kubelet-config\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398260 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-kubernetes\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-tuned\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398335 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-run-netns\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398316 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-sys-fs\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-device-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398399 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-device-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398421 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06f9552e-9efb-4688-8ed1-10c3e64f4249-cni-binary-copy\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398436 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398456 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysconfig\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398474 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-tmp\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398502 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6d6e1ef-fd1d-4a47-a28c-16b1343110b9-konnectivity-ca\") pod \"konnectivity-agent-tz5vp\" (UID: \"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9\") " pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398517 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-cni-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398536 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-cnibin\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398553 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-kubelet\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398566 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-hostroot\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6d6e1ef-fd1d-4a47-a28c-16b1343110b9-agent-certs\") pod \"konnectivity-agent-tz5vp\" (UID: \"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9\") " pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.398661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-socket-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398657 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e982204-a919-4ae0-9ea7-1af013a9324d-hosts-file\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398706 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3e982204-a919-4ae0-9ea7-1af013a9324d-hosts-file\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398733 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-os-release\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398759 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfppn\" (UniqueName: \"kubernetes.io/projected/b96d29f3-79ce-4860-92b5-3e055377dbf2-kube-api-access-rfppn\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398761 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-socket-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398781 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-daemon-config\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398808 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398832 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-cni-netd\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398851 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovn-node-metrics-cert\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398872 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-run\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398887 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-lib-modules\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398901 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msnrd\" (UniqueName: \"kubernetes.io/projected/f31c226b-95f7-4a3f-9b94-476f82172154-kube-api-access-msnrd\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-ovn\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398946 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.398990 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovnkube-config\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399006 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 23:04:32.399361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8n78\" (UniqueName: \"kubernetes.io/projected/3e982204-a919-4ae0-9ea7-1af013a9324d-kube-api-access-h8n78\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399032 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d6d6e1ef-fd1d-4a47-a28c-16b1343110b9-konnectivity-ca\") pod \"konnectivity-agent-tz5vp\" (UID: \"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9\") " pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399056 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399083 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-multus-certs\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399104 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-systemd\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399117 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399133 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-registration-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399154 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399183 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-system-cni-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399207 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-os-release\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399257 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-cni-bin\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399258 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-registration-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399279 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-cni-multus\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399311 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-cni-bin\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399342 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpscq\" (UniqueName: \"kubernetes.io/projected/a09e14eb-e908-4ba5-afeb-c85498121530-kube-api-access-zpscq\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399374 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.399990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399398 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-netns\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399412 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-conf-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-slash\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399460 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysctl-d\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399481 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-systemd-units\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399504 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a09e14eb-e908-4ba5-afeb-c85498121530-host-slash\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399525 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afd99605-6853-41d1-8b9f-a4aa46695472-dbus\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399544 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399524 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399557 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a09e14eb-e908-4ba5-afeb-c85498121530-host-slash\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399559 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msqt\" (UniqueName: \"kubernetes.io/projected/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-kube-api-access-4msqt\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399599 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399632 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-k8s-cni-cncf-io\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399650 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-env-overrides\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399664 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlvr6\" (UniqueName: \"kubernetes.io/projected/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-kube-api-access-hlvr6\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399687 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e982204-a919-4ae0-9ea7-1af013a9324d-tmp-dir\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399711 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-kubelet\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.400585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399759 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afd99605-6853-41d1-8b9f-a4aa46695472-dbus\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399786 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b02cac67-ee6b-43a4-9151-6b30a4473f58-serviceca\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399822 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399875 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399893 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-socket-dir-parent\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399907 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-etc-kubernetes\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399916 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e982204-a919-4ae0-9ea7-1af013a9324d-tmp-dir\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bknv9\" (UniqueName: \"kubernetes.io/projected/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-kube-api-access-bknv9\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399944 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b02cac67-ee6b-43a4-9151-6b30a4473f58-host\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399967 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7psqb\" (UniqueName: \"kubernetes.io/projected/b02cac67-ee6b-43a4-9151-6b30a4473f58-kube-api-access-7psqb\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.399987 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a09e14eb-e908-4ba5-afeb-c85498121530-iptables-alerter-script\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400003 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400026 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-var-lib-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400060 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-node-log\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400105 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-log-socket\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400127 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysctl-conf\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.401077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400146 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-systemd\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.401534 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400187 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.401534 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.400310 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:32.401534 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.400396 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a09e14eb-e908-4ba5-afeb-c85498121530-iptables-alerter-script\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.401534 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.400416 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:32.900394231 +0000 UTC m=+3.122922432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:32.402393 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.402377 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d6d6e1ef-fd1d-4a47-a28c-16b1343110b9-agent-certs\") pod \"konnectivity-agent-tz5vp\" (UID: \"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9\") " pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.420325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.420302 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8n78\" (UniqueName: \"kubernetes.io/projected/3e982204-a919-4ae0-9ea7-1af013a9324d-kube-api-access-h8n78\") pod \"node-resolver-vb4f8\" (UID: \"3e982204-a919-4ae0-9ea7-1af013a9324d\") " pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.420418 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.420372 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4msqt\" (UniqueName: \"kubernetes.io/projected/b00b50fc-aa01-4d2a-8863-507cd48aa6bc-kube-api-access-4msqt\") pod \"aws-ebs-csi-driver-node-8h4tg\" (UID: \"b00b50fc-aa01-4d2a-8863-507cd48aa6bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.420546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.420529 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpscq\" (UniqueName: \"kubernetes.io/projected/a09e14eb-e908-4ba5-afeb-c85498121530-kube-api-access-zpscq\") pod \"iptables-alerter-2bqdg\" (UID: \"a09e14eb-e908-4ba5-afeb-c85498121530\") " pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.422046 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.422032 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:04:32.422087 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.422050 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:04:32.422087 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.422071 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:32.422145 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.422127 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:04:32.922113406 +0000 UTC m=+3.144641611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:32.500981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.500920 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnrd\" (UniqueName: \"kubernetes.io/projected/f31c226b-95f7-4a3f-9b94-476f82172154-kube-api-access-msnrd\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:32.500981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.500949 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-ovn\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.500981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.500965 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.500991 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovnkube-config\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501016 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501040 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-multus-certs\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501052 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-ovn\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501065 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501072 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-systemd\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501104 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-multus-certs\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501106 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-systemd\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501109 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501152 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-run-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501149 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.501187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501176 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-system-cni-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501256 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-os-release\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501277 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-cni-bin\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-cni-multus\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501318 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-cni-bin\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501323 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-system-cni-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501323 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-os-release\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501352 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-cni-bin\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501358 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-netns\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501377 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-cni-multus\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501393 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-netns\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501398 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-cni-bin\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501402 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-conf-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501442 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-slash\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501442 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-conf-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501463 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-slash\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501464 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysctl-d\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501486 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-systemd-units\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.501773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501505 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-k8s-cni-cncf-io\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501527 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-env-overrides\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlvr6\" (UniqueName: \"kubernetes.io/projected/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-kube-api-access-hlvr6\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501566 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-run-k8s-cni-cncf-io\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501567 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-systemd-units\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501574 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-kubelet\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501633 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b02cac67-ee6b-43a4-9151-6b30a4473f58-serviceca\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501636 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovnkube-config\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501648 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysctl-d\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501687 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501691 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501776 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-kubelet\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501820 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-socket-dir-parent\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501822 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501849 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-etc-kubernetes\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501891 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bknv9\" (UniqueName: \"kubernetes.io/projected/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-kube-api-access-bknv9\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.502465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501907 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-socket-dir-parent\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501911 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-etc-kubernetes\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b02cac67-ee6b-43a4-9151-6b30a4473f58-host\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501953 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-env-overrides\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501967 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7psqb\" (UniqueName: \"kubernetes.io/projected/b02cac67-ee6b-43a4-9151-6b30a4473f58-kube-api-access-7psqb\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.501999 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502016 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b02cac67-ee6b-43a4-9151-6b30a4473f58-host\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-var-lib-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502045 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b02cac67-ee6b-43a4-9151-6b30a4473f58-serviceca\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-node-log\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502083 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-node-log\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502100 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-var-lib-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502092 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-log-socket\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502136 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-log-socket\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502135 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysctl-conf\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502170 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-systemd\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502200 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovnkube-script-lib\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502245 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-systemd\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502253 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysctl-conf\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502272 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-modprobe-d\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502292 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-sys\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502314 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-var-lib-kubelet\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-host\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502345 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2rds\" (UniqueName: \"kubernetes.io/projected/06f9552e-9efb-4688-8ed1-10c3e64f4249-kube-api-access-l2rds\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502366 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-etc-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502391 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-system-cni-dir\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502397 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-modprobe-d\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502399 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-var-lib-kubelet\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502413 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-host\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502416 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-cnibin\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502451 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-cnibin\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502451 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-etc-openvswitch\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502465 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-system-cni-dir\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502474 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b96d29f3-79ce-4860-92b5-3e055377dbf2-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502436 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-sys\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.503998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502468 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-kubernetes\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502510 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-tuned\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502525 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-kubernetes\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502528 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-run-netns\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502549 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-run-netns\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502570 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06f9552e-9efb-4688-8ed1-10c3e64f4249-cni-binary-copy\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502597 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502626 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovnkube-script-lib\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502641 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysconfig\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502666 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-tmp\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502674 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502691 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-cni-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502714 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-cnibin\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502721 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-sysconfig\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502738 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-kubelet\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502759 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-cni-dir\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502760 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-hostroot\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502796 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-hostroot\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.504806 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502795 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-os-release\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502820 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-host-var-lib-kubelet\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502834 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfppn\" (UniqueName: \"kubernetes.io/projected/b96d29f3-79ce-4860-92b5-3e055377dbf2-kube-api-access-rfppn\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502864 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-daemon-config\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502879 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06f9552e-9efb-4688-8ed1-10c3e64f4249-cnibin\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502892 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502839 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b96d29f3-79ce-4860-92b5-3e055377dbf2-os-release\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502917 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-cni-netd\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502950 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovn-node-metrics-cert\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502953 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-host-cni-netd\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.502972 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-run\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.503014 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.503069 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06f9552e-9efb-4688-8ed1-10c3e64f4249-cni-binary-copy\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.503078 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:33.00306168 +0000 UTC m=+3.225589879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.503019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-lib-modules\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.503129 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-lib-modules\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.503130 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-run\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.505353 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.503414 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06f9552e-9efb-4688-8ed1-10c3e64f4249-multus-daemon-config\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.505847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.504600 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-tmp\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.505847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.504840 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-ovn-node-metrics-cert\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.505847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.504899 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-etc-tuned\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.520962 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.520938 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2rds\" (UniqueName: \"kubernetes.io/projected/06f9552e-9efb-4688-8ed1-10c3e64f4249-kube-api-access-l2rds\") pod \"multus-xdblm\" (UID: \"06f9552e-9efb-4688-8ed1-10c3e64f4249\") " pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.521836 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.521818 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlvr6\" (UniqueName: \"kubernetes.io/projected/b6d8224c-48f5-425f-ab68-e7b0fe99ef4b-kube-api-access-hlvr6\") pod \"ovnkube-node-q5hbn\" (UID: \"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.521969 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.521948 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfppn\" (UniqueName: \"kubernetes.io/projected/b96d29f3-79ce-4860-92b5-3e055377dbf2-kube-api-access-rfppn\") pod \"multus-additional-cni-plugins-b8j64\" (UID: \"b96d29f3-79ce-4860-92b5-3e055377dbf2\") " pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.523219 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.523202 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bknv9\" (UniqueName: \"kubernetes.io/projected/4b0ed41f-340e-4a2a-a0f7-ae528afccc0a-kube-api-access-bknv9\") pod \"tuned-mct7s\" (UID: \"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a\") " pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.523631 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.523599 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msnrd\" (UniqueName: \"kubernetes.io/projected/f31c226b-95f7-4a3f-9b94-476f82172154-kube-api-access-msnrd\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:32.525253 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.525238 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7psqb\" (UniqueName: \"kubernetes.io/projected/b02cac67-ee6b-43a4-9151-6b30a4473f58-kube-api-access-7psqb\") pod \"node-ca-l8tqd\" (UID: \"b02cac67-ee6b-43a4-9151-6b30a4473f58\") " pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.527184 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.527171 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2bqdg" Jul 01 23:04:32.532987 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.532970 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:32.537490 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.537473 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09e14eb_e908_4ba5_afeb_c85498121530.slice/crio-b19858a85b898bb836104096ec5ee6b157ea28902be2f9ed3d5addccb315f0e8 WatchSource:0}: Error finding container b19858a85b898bb836104096ec5ee6b157ea28902be2f9ed3d5addccb315f0e8: Status 404 returned error can't find the container with id b19858a85b898bb836104096ec5ee6b157ea28902be2f9ed3d5addccb315f0e8 Jul 01 23:04:32.541989 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.541972 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d6e1ef_fd1d_4a47_a28c_16b1343110b9.slice/crio-a5322c561a6a0af9c24199c349d01c808ecf32bf0f05d68666c98e1847903384 WatchSource:0}: Error finding container a5322c561a6a0af9c24199c349d01c808ecf32bf0f05d68666c98e1847903384: Status 404 returned error can't find the container with id a5322c561a6a0af9c24199c349d01c808ecf32bf0f05d68666c98e1847903384 Jul 01 23:04:32.543160 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.542947 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" Jul 01 23:04:32.547424 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.547405 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vb4f8" Jul 01 23:04:32.551187 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.551170 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00b50fc_aa01_4d2a_8863_507cd48aa6bc.slice/crio-52d02c7aa98bc2ca31f6f503301ae0fb120760721a353ed87e631b89048f73d2 WatchSource:0}: Error finding container 52d02c7aa98bc2ca31f6f503301ae0fb120760721a353ed87e631b89048f73d2: Status 404 returned error can't find the container with id 52d02c7aa98bc2ca31f6f503301ae0fb120760721a353ed87e631b89048f73d2 Jul 01 23:04:32.552051 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.552036 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8j64" Jul 01 23:04:32.555683 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.555668 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e982204_a919_4ae0_9ea7_1af013a9324d.slice/crio-776c2a5a6756f327cd5d11d68be35fbe2b7cbe32bc5c4620ee03ca3c2bfee2ae WatchSource:0}: Error finding container 776c2a5a6756f327cd5d11d68be35fbe2b7cbe32bc5c4620ee03ca3c2bfee2ae: Status 404 returned error can't find the container with id 776c2a5a6756f327cd5d11d68be35fbe2b7cbe32bc5c4620ee03ca3c2bfee2ae Jul 01 23:04:32.558047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.558022 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xdblm" Jul 01 23:04:32.561251 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.561232 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96d29f3_79ce_4860_92b5_3e055377dbf2.slice/crio-7ca88b4027261b3969f47174ed13f49b7929b3df312b09fc7cc20369aaeca4e6 WatchSource:0}: Error finding container 7ca88b4027261b3969f47174ed13f49b7929b3df312b09fc7cc20369aaeca4e6: Status 404 returned error can't find the container with id 7ca88b4027261b3969f47174ed13f49b7929b3df312b09fc7cc20369aaeca4e6 Jul 01 23:04:32.562640 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.562546 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:32.566652 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.566629 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f9552e_9efb_4688_8ed1_10c3e64f4249.slice/crio-8215589903c93a66be7f1d449a58735ee37a09e3fa7c9923bda4c3c013a8b22d WatchSource:0}: Error finding container 8215589903c93a66be7f1d449a58735ee37a09e3fa7c9923bda4c3c013a8b22d: Status 404 returned error can't find the container with id 8215589903c93a66be7f1d449a58735ee37a09e3fa7c9923bda4c3c013a8b22d Jul 01 23:04:32.567655 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.567636 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mct7s" Jul 01 23:04:32.571339 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.571318 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8224c_48f5_425f_ab68_e7b0fe99ef4b.slice/crio-65a7b32fb11d31a0670994154c5f078570efbf7f6817e74a298b6a5962d552b1 WatchSource:0}: Error finding container 65a7b32fb11d31a0670994154c5f078570efbf7f6817e74a298b6a5962d552b1: Status 404 returned error can't find the container with id 65a7b32fb11d31a0670994154c5f078570efbf7f6817e74a298b6a5962d552b1 Jul 01 23:04:32.572127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.572095 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l8tqd" Jul 01 23:04:32.577258 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.577231 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0ed41f_340e_4a2a_a0f7_ae528afccc0a.slice/crio-348c13e434f6d614476372e6432705138b8ea5d6ff16a0fbff4d0fabf34940da WatchSource:0}: Error finding container 348c13e434f6d614476372e6432705138b8ea5d6ff16a0fbff4d0fabf34940da: Status 404 returned error can't find the container with id 348c13e434f6d614476372e6432705138b8ea5d6ff16a0fbff4d0fabf34940da Jul 01 23:04:32.583790 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:04:32.583746 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02cac67_ee6b_43a4_9151_6b30a4473f58.slice/crio-ca83ee0a88b17742ea2f7b758c78c8511e85a858d3752ce3ef692ac0ee48fd2a WatchSource:0}: Error finding container ca83ee0a88b17742ea2f7b758c78c8511e85a858d3752ce3ef692ac0ee48fd2a: Status 404 returned error can't find the container with id ca83ee0a88b17742ea2f7b758c78c8511e85a858d3752ce3ef692ac0ee48fd2a Jul 01 23:04:32.906399 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.906322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:32.906565 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.906511 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:32.906653 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:32.906583 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:33.906564801 +0000 UTC m=+4.129092994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:32.921619 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.921553 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l8tqd" event={"ID":"b02cac67-ee6b-43a4-9151-6b30a4473f58","Type":"ContainerStarted","Data":"ca83ee0a88b17742ea2f7b758c78c8511e85a858d3752ce3ef692ac0ee48fd2a"} Jul 01 23:04:32.922884 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.922848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mct7s" event={"ID":"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a","Type":"ContainerStarted","Data":"348c13e434f6d614476372e6432705138b8ea5d6ff16a0fbff4d0fabf34940da"} Jul 01 23:04:32.924466 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.924431 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"65a7b32fb11d31a0670994154c5f078570efbf7f6817e74a298b6a5962d552b1"} Jul 01 23:04:32.926368 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.926340 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xdblm" event={"ID":"06f9552e-9efb-4688-8ed1-10c3e64f4249","Type":"ContainerStarted","Data":"8215589903c93a66be7f1d449a58735ee37a09e3fa7c9923bda4c3c013a8b22d"} Jul 01 23:04:32.927979 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.927940 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vb4f8" event={"ID":"3e982204-a919-4ae0-9ea7-1af013a9324d","Type":"ContainerStarted","Data":"776c2a5a6756f327cd5d11d68be35fbe2b7cbe32bc5c4620ee03ca3c2bfee2ae"} Jul 01 23:04:32.929621 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.929537 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" event={"ID":"b00b50fc-aa01-4d2a-8863-507cd48aa6bc","Type":"ContainerStarted","Data":"52d02c7aa98bc2ca31f6f503301ae0fb120760721a353ed87e631b89048f73d2"} Jul 01 23:04:32.931303 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.931279 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" event={"ID":"cec0e47b4d0f2d635d3199e833873acd","Type":"ContainerStarted","Data":"2b437f0fc0df88cc24ed6767a09ae3dd11130612f505c981645cae33b249497f"} Jul 01 23:04:32.933031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.933009 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerStarted","Data":"7ca88b4027261b3969f47174ed13f49b7929b3df312b09fc7cc20369aaeca4e6"} Jul 01 23:04:32.937324 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.937307 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tz5vp" event={"ID":"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9","Type":"ContainerStarted","Data":"a5322c561a6a0af9c24199c349d01c808ecf32bf0f05d68666c98e1847903384"} Jul 01 23:04:32.940686 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.940620 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2bqdg" event={"ID":"a09e14eb-e908-4ba5-afeb-c85498121530","Type":"ContainerStarted","Data":"b19858a85b898bb836104096ec5ee6b157ea28902be2f9ed3d5addccb315f0e8"} Jul 01 23:04:32.943861 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:32.943835 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" event={"ID":"f2ca051fa183ca1da2ae0341414c5a72","Type":"ContainerStarted","Data":"b59bc30137470850caa133a8d677b44a48c71a76907ae0376a140cca6368ad1f"} Jul 01 23:04:33.007569 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.007529 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:33.007752 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.007632 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:33.007752 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.007747 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:33.007869 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.007819 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:34.007800059 +0000 UTC m=+4.230328249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:33.007928 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.007911 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:04:33.007928 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.007925 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:04:33.008026 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.007938 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:33.008026 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.007977 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:04:34.007964078 +0000 UTC m=+4.230492269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:33.287540 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.287389 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 22:59:32 +0000 UTC" deadline="2028-01-05 10:02:31.783759325 +0000 UTC" Jul 01 23:04:33.287540 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.287449 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13258h57m58.496314123s" Jul 01 23:04:33.915134 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.915099 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:33.915293 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.915271 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:33.915347 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.915335 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:35.915315926 +0000 UTC m=+6.137844123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:33.916653 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.916624 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:33.916759 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.916733 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:33.916832 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.916817 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:33.916918 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.916893 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:33.916961 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:33.916952 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:33.917053 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:33.917031 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:34.016518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:34.016205 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:34.016518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:34.016280 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:34.016518 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:34.016423 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:04:34.016518 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:34.016439 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:04:34.016518 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:34.016451 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:34.016518 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:34.016507 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:04:36.016490646 +0000 UTC m=+6.239018854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:34.016969 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:34.016568 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:34.016969 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:34.016598 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:36.016588565 +0000 UTC m=+6.239116753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:34.193954 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:34.193879 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 23:04:34.287644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:34.287587 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 22:59:32 +0000 UTC" deadline="2028-03-24 08:52:11.459150384 +0000 UTC" Jul 01 23:04:34.287644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:34.287642 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15153h47m37.171513995s" Jul 01 23:04:35.916671 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:35.916635 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:35.917113 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:35.916784 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:35.917113 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:35.916869 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:35.917113 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:35.916946 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:35.917113 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:35.916995 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:35.917113 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:35.917086 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:35.932287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:35.932255 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:35.932444 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:35.932421 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:35.932504 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:35.932497 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:39.932476973 +0000 UTC m=+10.155005166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:36.033120 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:36.033085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:36.033304 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:36.033179 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:36.033368 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:36.033322 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:36.033425 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:36.033387 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:40.033368277 +0000 UTC m=+10.255896486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:36.033483 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:36.033476 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:04:36.033551 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:36.033494 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:04:36.033551 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:36.033507 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:36.033551 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:36.033544 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:04:40.033532704 +0000 UTC m=+10.256060906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:37.917175 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:37.917139 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:37.917638 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:37.917185 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:37.917638 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:37.917296 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:37.917638 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:37.917362 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:37.917638 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:37.917415 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:37.917638 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:37.917493 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:39.917347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:39.916875 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:39.917347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:39.916916 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:39.917347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:39.916988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:39.917347 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:39.917007 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:39.917347 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:39.917106 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:39.917347 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:39.917191 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:39.965820 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:39.965786 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:39.966005 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:39.965901 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:39.966005 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:39.965964 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:47.96594199 +0000 UTC m=+18.188470200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:40.066833 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:40.066900 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:40.067038 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:40.067055 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:40.067076 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:40.067088 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:40.067117 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:04:48.067092815 +0000 UTC m=+18.289621004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:40.067439 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:40.067134 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:04:48.067127431 +0000 UTC m=+18.289655618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:41.916771 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:41.916690 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:41.917203 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:41.916833 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:41.917284 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:41.917213 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:41.917338 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:41.917308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:41.917395 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:41.917357 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:41.917447 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:41.917426 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:43.916928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:43.916896 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:43.917287 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:43.917012 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:43.917287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:43.916895 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:43.917287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:43.916896 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:43.917287 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:43.917128 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:43.917287 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:43.917197 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:45.917147 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:45.917116 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:45.917147 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:45.917134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:45.917647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:45.917224 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:45.917647 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:45.917237 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:45.917647 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:45.917308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:45.917647 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:45.917360 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:47.917101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:47.917064 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:47.917598 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:47.917122 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:47.917598 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:47.917063 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:47.917598 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:47.917204 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:47.917598 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:47.917277 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:47.917598 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:47.917359 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:48.026012 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:48.025979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:48.026149 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.026097 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:48.026149 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.026143 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:05:04.026130673 +0000 UTC m=+34.248658861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:04:48.126540 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:48.126504 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:48.126706 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:48.126557 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:48.126706 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.126675 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:48.126797 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.126745 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:05:04.126726605 +0000 UTC m=+34.349254811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:04:48.126797 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.126679 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:04:48.126797 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.126786 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:04:48.126797 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.126798 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:48.126950 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:48.126843 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:05:04.126829489 +0000 UTC m=+34.349357682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:04:49.916928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:49.916892 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:49.917344 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:49.916903 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:49.917344 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:49.917016 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:49.917344 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:49.916909 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:49.917344 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:49.917093 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:49.917344 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:49.917173 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:51.917315 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:51.917282 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:51.917771 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:51.917384 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:51.917771 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:51.917390 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:51.917771 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:51.917402 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:51.917771 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:51.917506 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:51.917771 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:51.917593 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:53.002254 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.001965 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mct7s" event={"ID":"4b0ed41f-340e-4a2a-a0f7-ae528afccc0a","Type":"ContainerStarted","Data":"69bb3fcfa4572fc7d02dccce417e21b5994302d1f016cdf7dcac2d3dcfa507c3"} Jul 01 23:04:53.004492 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.004463 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xdblm" event={"ID":"06f9552e-9efb-4688-8ed1-10c3e64f4249","Type":"ContainerStarted","Data":"991efaa309bd7613939141c2bf90bc231e2e8bafffce7539d4cf485e96e78a25"} Jul 01 23:04:53.008976 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.008946 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" event={"ID":"cec0e47b4d0f2d635d3199e833873acd","Type":"ContainerStarted","Data":"0c57f764b616f9af125404861a8268c41af55428caaa9d622b01668b67fccf1f"} Jul 01 23:04:53.022970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.022911 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mct7s" podStartSLOduration=2.038127472 podStartE2EDuration="22.022899323s" totalImagesPullingTime="19.984771851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:53.022016226 +0000 UTC m=+23.244544451" watchObservedRunningTime="2026-07-01 23:04:53.022899323 +0000 UTC m=+23.245427533" Jul 01 23:04:53.037581 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.037524 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-234.ec2.internal" podStartSLOduration=22.0375086 podStartE2EDuration="22.0375086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:53.036987703 +0000 UTC m=+23.259515913" watchObservedRunningTime="2026-07-01 23:04:53.0375086 +0000 UTC m=+23.260036810" Jul 01 23:04:53.056474 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.056423 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xdblm" podStartSLOduration=2.049235375 podStartE2EDuration="22.056408862s" totalImagesPullingTime="20.007173487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:53.055857673 +0000 UTC m=+23.278385906" watchObservedRunningTime="2026-07-01 23:04:53.056408862 +0000 UTC m=+23.278937072" Jul 01 23:04:53.916945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.916764 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:53.917087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.916807 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:53.917087 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:53.917021 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:53.917169 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:53.917094 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:53.917169 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:53.916805 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:53.917241 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:53.917166 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:54.014032 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014003 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:04:54.014662 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014305 2569 generic.go:356] "Generic (PLEG): container finished" podID="b6d8224c-48f5-425f-ab68-e7b0fe99ef4b" containerID="c4f87b7fddc6c7cba09a374e6d596d7772db28c076eecd5d454f471ad816462b" exitCode=1 Jul 01 23:04:54.014662 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014380 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"e3e76ee350816b31249bbe6d01edb47b8aa4698862d61bb2f65df5a1ac57eb67"} Jul 01 23:04:54.014662 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014447 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"5b5de3eb1b3d981181a27cd5f62641a6b121db24610518ab10b96ee7599c2c27"} Jul 01 23:04:54.014826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"43b24cdb0d276b90f6348afa891b711e009eaf9ff686a8dc1a0802e984ae3cc3"} Jul 01 23:04:54.014826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014725 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"c92da7798ac730da21fe3dc2e07a6530cf812719296173689468fe65f50f90d1"} Jul 01 23:04:54.014826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014757 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerDied","Data":"c4f87b7fddc6c7cba09a374e6d596d7772db28c076eecd5d454f471ad816462b"} Jul 01 23:04:54.014826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.014794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"0388e7d4170297c9195c20bf3f452a90872143a81afb3782b1973144f102f407"} Jul 01 23:04:54.015679 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.015659 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vb4f8" event={"ID":"3e982204-a919-4ae0-9ea7-1af013a9324d","Type":"ContainerStarted","Data":"41bbfd602f13e8807e771473f03017225894f5c268e92e6d83b997d50f5291f4"} Jul 01 23:04:54.017136 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.017111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" event={"ID":"b00b50fc-aa01-4d2a-8863-507cd48aa6bc","Type":"ContainerStarted","Data":"534a7479f89111b672e3c6b682901087437968c32790ea5d84a27e00f8793c83"} Jul 01 23:04:54.018445 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.018424 2569 generic.go:356] "Generic (PLEG): container finished" podID="b96d29f3-79ce-4860-92b5-3e055377dbf2" containerID="d7c68befdb9403d816f03632cc9dcefe13e614f02a4a2bf920450b0ffbdcd3b9" exitCode=0 Jul 01 23:04:54.018535 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.018491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerDied","Data":"d7c68befdb9403d816f03632cc9dcefe13e614f02a4a2bf920450b0ffbdcd3b9"} Jul 01 23:04:54.022158 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.022129 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tz5vp" event={"ID":"d6d6e1ef-fd1d-4a47-a28c-16b1343110b9","Type":"ContainerStarted","Data":"6592a192b517f637b666c9ed3e304552a438785bb2a564ce9b89c55ca00f916a"} Jul 01 23:04:54.023814 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.023790 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2bqdg" event={"ID":"a09e14eb-e908-4ba5-afeb-c85498121530","Type":"ContainerStarted","Data":"6b5499ef1d28d228613d5e3df2c658b881fcda0f9b0dc255222d157878f6b0e7"} Jul 01 23:04:54.025427 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.025405 2569 generic.go:356] "Generic (PLEG): container finished" podID="f2ca051fa183ca1da2ae0341414c5a72" containerID="ae192c39d162e9a2e5e6eefc88ff86dc03b65f1282f522bc1247476ee5dcc707" exitCode=0 Jul 01 23:04:54.025533 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.025499 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" event={"ID":"f2ca051fa183ca1da2ae0341414c5a72","Type":"ContainerDied","Data":"ae192c39d162e9a2e5e6eefc88ff86dc03b65f1282f522bc1247476ee5dcc707"} Jul 01 23:04:54.026994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.026971 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l8tqd" event={"ID":"b02cac67-ee6b-43a4-9151-6b30a4473f58","Type":"ContainerStarted","Data":"6d217ee519134f998a78ec3a3286994c7138dbdb9019990009586061cc10876e"} Jul 01 23:04:54.032551 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.032512 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vb4f8" podStartSLOduration=3.052684059 podStartE2EDuration="23.032497179s" totalImagesPullingTime="19.97981312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:54.03240097 +0000 UTC m=+24.254929205" watchObservedRunningTime="2026-07-01 23:04:54.032497179 +0000 UTC m=+24.255025390" Jul 01 23:04:54.070343 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.070303 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2bqdg" podStartSLOduration=3.044746205 podStartE2EDuration="23.070286439s" totalImagesPullingTime="20.025540234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:54.070170123 +0000 UTC m=+24.292698359" watchObservedRunningTime="2026-07-01 23:04:54.070286439 +0000 UTC m=+24.292814650" Jul 01 23:04:54.087481 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.087431 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tz5vp" podStartSLOduration=3.094284174 podStartE2EDuration="23.087417341s" totalImagesPullingTime="19.993133167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:54.086917355 +0000 UTC m=+24.309445590" watchObservedRunningTime="2026-07-01 23:04:54.087417341 +0000 UTC m=+24.309945542" Jul 01 23:04:54.121366 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.121316 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l8tqd" podStartSLOduration=3.169461264 podStartE2EDuration="23.121297291s" totalImagesPullingTime="19.951836027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:54.103494493 +0000 UTC m=+24.326022707" watchObservedRunningTime="2026-07-01 23:04:54.121297291 +0000 UTC m=+24.343825503" Jul 01 23:04:54.777486 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.777453 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 23:04:54.932510 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.932296 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T23:04:54.777476651Z","UUID":"b1ade8ac-02c9-4173-b855-1eb93971fbcc","Handler":null,"Name":"","Endpoint":""} Jul 01 23:04:54.933991 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.933974 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 23:04:54.934113 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:54.934010 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 23:04:55.031405 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:55.031371 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" event={"ID":"f2ca051fa183ca1da2ae0341414c5a72","Type":"ContainerStarted","Data":"08d77a5c6445109a6fcb0d16eae6e5975a7d7fbfc167cd7d041e3db774aa6994"} Jul 01 23:04:55.033517 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:55.033491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" event={"ID":"b00b50fc-aa01-4d2a-8863-507cd48aa6bc","Type":"ContainerStarted","Data":"3cb70a3060506bed1846daa43f98875da0ee88806f17de49d8c2f8b1e893c4e9"} Jul 01 23:04:55.064399 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:55.064347 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-234.ec2.internal" podStartSLOduration=24.064333458 podStartE2EDuration="24.064333458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:55.063393504 +0000 UTC m=+25.285921739" watchObservedRunningTime="2026-07-01 23:04:55.064333458 +0000 UTC m=+25.286861667" Jul 01 23:04:55.917190 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:55.917165 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:55.917329 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:55.917200 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:55.917329 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:55.917165 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:55.917329 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:55.917291 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:55.917578 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:55.917347 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:55.917578 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:55.917466 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:56.039846 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:56.039822 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:04:56.040324 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:56.040293 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"1108e8a89991d0067240d31f3cb3ddc9f3488f4a681d155d410f6edbb773f7e1"} Jul 01 23:04:56.497585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:56.497511 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:56.498358 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:56.498336 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:57.045259 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.045218 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" event={"ID":"b00b50fc-aa01-4d2a-8863-507cd48aa6bc","Type":"ContainerStarted","Data":"b5de8fbe213cc036bf590d26cf6df1ef780a849d060be1f844cf052a4e8b15cf"} Jul 01 23:04:57.045770 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.045531 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:57.046136 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.046105 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tz5vp" Jul 01 23:04:57.067129 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.067087 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8h4tg" podStartSLOduration=11.908296086 podStartE2EDuration="26.067073865s" totalImagesPullingTime="14.158777779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:04:57.066943903 +0000 UTC m=+27.289472148" watchObservedRunningTime="2026-07-01 23:04:57.067073865 +0000 UTC m=+27.289602074" Jul 01 23:04:57.916834 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.916789 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:57.916994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.916798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:57.916994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:57.916923 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:57.916994 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:57.916889 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:04:57.916994 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:57.916976 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:57.917198 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:57.917091 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:59.053424 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.053262 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:04:59.054026 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.053807 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"f40ed2106a2177e9b6c4c3e172e54e7a1939d7e4220b1bb1cb5927a40648794c"} Jul 01 23:04:59.054159 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.054139 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:59.054268 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.054253 2569 scope.go:117] "RemoveContainer" containerID="c4f87b7fddc6c7cba09a374e6d596d7772db28c076eecd5d454f471ad816462b" Jul 01 23:04:59.055648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.055622 2569 generic.go:356] "Generic (PLEG): container finished" podID="b96d29f3-79ce-4860-92b5-3e055377dbf2" containerID="f508629cdf6e32e2de38364fca065d48e78cbf301413749bbaea855430e69f92" exitCode=0 Jul 01 23:04:59.055741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.055655 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerDied","Data":"f508629cdf6e32e2de38364fca065d48e78cbf301413749bbaea855430e69f92"} Jul 01 23:04:59.069442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.069424 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:04:59.917394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.917356 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:04:59.917648 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:59.917477 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:04:59.917648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.917366 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:04:59.917648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:04:59.917356 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:04:59.917648 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:59.917559 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:04:59.917852 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:04:59.917717 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:05:00.062457 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.062433 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:05:00.062952 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.062882 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" event={"ID":"b6d8224c-48f5-425f-ab68-e7b0fe99ef4b","Type":"ContainerStarted","Data":"d6df8baf6d5fe2ff0a5acffa954916d665c988f166003cf8959585425e9b41cb"} Jul 01 23:05:00.063122 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.063090 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:05:00.063222 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.063175 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:05:00.079778 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.079755 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:05:00.092700 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.092649 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" podStartSLOduration=8.636707561 podStartE2EDuration="29.092628172s" totalImagesPullingTime="20.455920611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:00.091029165 +0000 UTC m=+30.313557427" watchObservedRunningTime="2026-07-01 23:05:00.092628172 +0000 UTC m=+30.315156381" Jul 01 23:05:00.805181 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.804930 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjqgl"] Jul 01 23:05:00.805338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.805214 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:00.805338 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:00.805290 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:05:00.806677 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.806656 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4pz9m"] Jul 01 23:05:00.806772 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.806747 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:00.806836 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:00.806820 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:05:00.810746 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.810722 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ghqjk"] Jul 01 23:05:00.810851 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:00.810818 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:00.810930 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:00.810909 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:05:01.006791 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:01.006765 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vb4f8_3e982204-a919-4ae0-9ea7-1af013a9324d/dns-node-resolver/0.log" Jul 01 23:05:01.066704 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:01.066669 2569 generic.go:356] "Generic (PLEG): container finished" podID="b96d29f3-79ce-4860-92b5-3e055377dbf2" containerID="e21ef81f13e329dbd3e76c748a487309f3a8e86707dde0d77142b2fdd172717c" exitCode=0 Jul 01 23:05:01.067153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:01.066747 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerDied","Data":"e21ef81f13e329dbd3e76c748a487309f3a8e86707dde0d77142b2fdd172717c"} Jul 01 23:05:01.767760 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:01.767716 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l8tqd_b02cac67-ee6b-43a4-9151-6b30a4473f58/node-ca/0.log" Jul 01 23:05:02.917476 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:02.917406 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:02.917988 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:02.917407 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:02.917988 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:02.917519 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:05:02.917988 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:02.917407 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:02.917988 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:02.917585 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:05:02.917988 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:02.917707 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:05:03.072974 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:03.072943 2569 generic.go:356] "Generic (PLEG): container finished" podID="b96d29f3-79ce-4860-92b5-3e055377dbf2" containerID="53576b26b46d28d1ac5971dbf8a4e2e8c5818a081c5d58959216ecb2c7dab760" exitCode=0 Jul 01 23:05:03.073131 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:03.072982 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerDied","Data":"53576b26b46d28d1ac5971dbf8a4e2e8c5818a081c5d58959216ecb2c7dab760"} Jul 01 23:05:04.045183 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:04.045143 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:04.045665 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.045291 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 23:05:04.045665 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.045362 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret podName:afd99605-6853-41d1-8b9f-a4aa46695472 nodeName:}" failed. No retries permitted until 2026-07-01 23:05:36.045343438 +0000 UTC m=+66.267871626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret") pod "global-pull-secret-syncer-xjqgl" (UID: "afd99605-6853-41d1-8b9f-a4aa46695472") : object "kube-system"/"original-pull-secret" not registered Jul 01 23:05:04.146272 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:04.146231 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:04.146446 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:04.146299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:04.146446 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.146397 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:05:04.146555 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.146465 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 23:05:04.146555 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.146476 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs podName:f31c226b-95f7-4a3f-9b94-476f82172154 nodeName:}" failed. No retries permitted until 2026-07-01 23:05:36.146452418 +0000 UTC m=+66.368980620 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs") pod "network-metrics-daemon-ghqjk" (UID: "f31c226b-95f7-4a3f-9b94-476f82172154") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 23:05:04.146555 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.146484 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 23:05:04.146555 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.146510 2569 projected.go:196] Error preparing data for projected volume kube-api-access-7klzx for pod openshift-network-diagnostics/network-check-target-4pz9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:05:04.146777 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.146563 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx podName:2ffdec5f-ae85-4322-973f-bcf570740a6f nodeName:}" failed. No retries permitted until 2026-07-01 23:05:36.1465484 +0000 UTC m=+66.369076603 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7klzx" (UniqueName: "kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx") pod "network-check-target-4pz9m" (UID: "2ffdec5f-ae85-4322-973f-bcf570740a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 23:05:04.917239 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:04.916889 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:04.917239 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:04.916889 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:04.917239 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.916996 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pz9m" podUID="2ffdec5f-ae85-4322-973f-bcf570740a6f" Jul 01 23:05:04.917239 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:04.916889 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:04.917239 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.917075 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjqgl" podUID="afd99605-6853-41d1-8b9f-a4aa46695472" Jul 01 23:05:04.917239 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:04.917167 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghqjk" podUID="f31c226b-95f7-4a3f-9b94-476f82172154" Jul 01 23:05:06.439272 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.439057 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-234.ec2.internal" event="NodeReady" Jul 01 23:05:06.439730 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.439374 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 23:05:06.917161 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.917082 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:06.917161 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.917128 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:06.917161 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.917086 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:06.920713 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.920672 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 23:05:06.920713 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.920698 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 23:05:06.920918 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.920761 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 23:05:06.920982 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.920958 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6x5m9\"" Jul 01 23:05:06.921043 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.920996 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 23:05:06.921142 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:06.921124 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-96nlh\"" Jul 01 23:05:08.086895 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:08.086866 2569 generic.go:356] "Generic (PLEG): container finished" podID="b96d29f3-79ce-4860-92b5-3e055377dbf2" containerID="1266f71537128c7c619089822800516d5a734a146f3053190d48ab53c1a6fa8f" exitCode=0 Jul 01 23:05:08.087330 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:08.086923 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerDied","Data":"1266f71537128c7c619089822800516d5a734a146f3053190d48ab53c1a6fa8f"} Jul 01 23:05:09.091622 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:09.091572 2569 generic.go:356] "Generic (PLEG): container finished" podID="b96d29f3-79ce-4860-92b5-3e055377dbf2" containerID="be1ee3b3d09dde78a86135a398966030154ed00bf0ca996ae95aac3c7c39adab" exitCode=0 Jul 01 23:05:09.092081 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:09.091631 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerDied","Data":"be1ee3b3d09dde78a86135a398966030154ed00bf0ca996ae95aac3c7c39adab"} Jul 01 23:05:10.096688 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:10.096655 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8j64" event={"ID":"b96d29f3-79ce-4860-92b5-3e055377dbf2","Type":"ContainerStarted","Data":"579b951726b7d1aa41b452e827d569220a4f17c7cb41f6521d0f4bea11daa32b"} Jul 01 23:05:10.122601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:10.122561 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b8j64" podStartSLOduration=7.994842242 podStartE2EDuration="39.122547718s" totalImagesPullingTime="31.127705476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:10.122074279 +0000 UTC m=+40.344602488" watchObservedRunningTime="2026-07-01 23:05:10.122547718 +0000 UTC m=+40.345075982" Jul 01 23:05:27.768090 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.768052 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nt6sb"] Jul 01 23:05:27.770294 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.770277 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8msw2"] Jul 01 23:05:27.770407 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.770388 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:27.772895 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.772871 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:27.773256 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.773225 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 23:05:27.773378 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.773291 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 23:05:27.773822 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.773799 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8t5h6\"" Jul 01 23:05:27.776307 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.776287 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wrcbb\"" Jul 01 23:05:27.776395 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.776380 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 23:05:27.777526 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.777369 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 23:05:27.777884 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.777869 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 23:05:27.777987 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.777933 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 23:05:27.803248 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.803214 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nt6sb"] Jul 01 23:05:27.803974 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.803956 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8msw2"] Jul 01 23:05:27.866330 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.866298 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7hsss"] Jul 01 23:05:27.867994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.867973 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:27.870528 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.870504 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 23:05:27.870659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.870642 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 23:05:27.871146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.871132 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 23:05:27.871272 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.871258 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sp6fq\"" Jul 01 23:05:27.882789 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.882772 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7hsss"] Jul 01 23:05:27.900231 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900208 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-crio-socket\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:27.900348 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900236 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqz59\" (UniqueName: \"kubernetes.io/projected/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-kube-api-access-xqz59\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:27.900348 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900255 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqjvh\" (UniqueName: \"kubernetes.io/projected/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-kube-api-access-jqjvh\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:27.900348 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900274 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:27.900348 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900310 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-data-volume\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:27.900512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900354 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-config-volume\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:27.900512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900390 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:27.900512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900431 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-metrics-tls\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:27.900512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.900458 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-tmp-dir\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:27.963818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.963791 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ddf54f849-qx9jj"] Jul 01 23:05:27.966132 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.966117 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:27.968393 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.968359 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 23:05:27.968632 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.968599 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 23:05:27.968700 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.968686 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 23:05:27.968981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.968960 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kngfc\"" Jul 01 23:05:27.973358 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.973340 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 23:05:27.980976 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:27.980958 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ddf54f849-qx9jj"] Jul 01 23:05:28.001461 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-metrics-tls\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.001564 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001466 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgtjk\" (UniqueName: \"kubernetes.io/projected/07cd1d83-bd84-4673-afbc-dea22ae373e1-kube-api-access-pgtjk\") pod \"ingress-canary-7hsss\" (UID: \"07cd1d83-bd84-4673-afbc-dea22ae373e1\") " pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.001564 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001490 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-tmp-dir\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.001564 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001505 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07cd1d83-bd84-4673-afbc-dea22ae373e1-cert\") pod \"ingress-canary-7hsss\" (UID: \"07cd1d83-bd84-4673-afbc-dea22ae373e1\") " pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.001564 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001535 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-crio-socket\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.001761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001574 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqz59\" (UniqueName: \"kubernetes.io/projected/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-kube-api-access-xqz59\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.001761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001591 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqjvh\" (UniqueName: \"kubernetes.io/projected/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-kube-api-access-jqjvh\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.001761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.001761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001667 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-data-volume\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.001761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001676 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-crio-socket\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.001761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001711 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-config-volume\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.001965 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001843 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-tmp-dir\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.001965 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001867 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.002069 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.001988 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-data-volume\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.002292 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.002273 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-config-volume\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.002356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.002337 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.005481 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.005460 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.005587 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.005508 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-metrics-tls\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.016061 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.016033 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqz59\" (UniqueName: \"kubernetes.io/projected/d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d-kube-api-access-xqz59\") pod \"dns-default-nt6sb\" (UID: \"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d\") " pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.016160 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.016102 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqjvh\" (UniqueName: \"kubernetes.io/projected/bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23-kube-api-access-jqjvh\") pod \"insights-runtime-extractor-8msw2\" (UID: \"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23\") " pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.081116 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.081093 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:28.085343 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.085328 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8msw2" Jul 01 23:05:28.103040 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103009 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73ac79ed-a1fa-4ce2-871a-267fde34f392-ca-trust-extracted\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103156 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103049 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-bound-sa-token\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103156 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103077 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07cd1d83-bd84-4673-afbc-dea22ae373e1-cert\") pod \"ingress-canary-7hsss\" (UID: \"07cd1d83-bd84-4673-afbc-dea22ae373e1\") " pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.103156 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103143 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73ac79ed-a1fa-4ce2-871a-267fde34f392-installation-pull-secrets\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103298 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103201 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73ac79ed-a1fa-4ce2-871a-267fde34f392-registry-certificates\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103298 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103243 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73ac79ed-a1fa-4ce2-871a-267fde34f392-image-registry-private-configuration\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103298 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103261 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-registry-tls\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103298 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103282 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7sb8\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-kube-api-access-p7sb8\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103474 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103308 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73ac79ed-a1fa-4ce2-871a-267fde34f392-trusted-ca\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.103474 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.103392 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgtjk\" (UniqueName: \"kubernetes.io/projected/07cd1d83-bd84-4673-afbc-dea22ae373e1-kube-api-access-pgtjk\") pod \"ingress-canary-7hsss\" (UID: \"07cd1d83-bd84-4673-afbc-dea22ae373e1\") " pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.105408 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.105390 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07cd1d83-bd84-4673-afbc-dea22ae373e1-cert\") pod \"ingress-canary-7hsss\" (UID: \"07cd1d83-bd84-4673-afbc-dea22ae373e1\") " pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.112045 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.112025 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgtjk\" (UniqueName: \"kubernetes.io/projected/07cd1d83-bd84-4673-afbc-dea22ae373e1-kube-api-access-pgtjk\") pod \"ingress-canary-7hsss\" (UID: \"07cd1d83-bd84-4673-afbc-dea22ae373e1\") " pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.176894 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.176863 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7hsss" Jul 01 23:05:28.204511 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204453 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73ac79ed-a1fa-4ce2-871a-267fde34f392-image-registry-private-configuration\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.204682 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204658 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-registry-tls\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.204757 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204708 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7sb8\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-kube-api-access-p7sb8\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.204757 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204746 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73ac79ed-a1fa-4ce2-871a-267fde34f392-trusted-ca\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.204869 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204800 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73ac79ed-a1fa-4ce2-871a-267fde34f392-ca-trust-extracted\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.204869 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204816 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-bound-sa-token\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.204869 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204845 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73ac79ed-a1fa-4ce2-871a-267fde34f392-installation-pull-secrets\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.205010 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.204908 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73ac79ed-a1fa-4ce2-871a-267fde34f392-registry-certificates\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.206165 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.206080 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73ac79ed-a1fa-4ce2-871a-267fde34f392-ca-trust-extracted\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.206670 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.206535 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73ac79ed-a1fa-4ce2-871a-267fde34f392-registry-certificates\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.207995 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.207940 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73ac79ed-a1fa-4ce2-871a-267fde34f392-trusted-ca\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.208393 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.208346 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-registry-tls\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.208580 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.208557 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73ac79ed-a1fa-4ce2-871a-267fde34f392-image-registry-private-configuration\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.208774 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.208558 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73ac79ed-a1fa-4ce2-871a-267fde34f392-installation-pull-secrets\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.214368 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.214346 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-bound-sa-token\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.214569 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.214550 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7sb8\" (UniqueName: \"kubernetes.io/projected/73ac79ed-a1fa-4ce2-871a-267fde34f392-kube-api-access-p7sb8\") pod \"image-registry-ddf54f849-qx9jj\" (UID: \"73ac79ed-a1fa-4ce2-871a-267fde34f392\") " pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.224772 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.224746 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8msw2"] Jul 01 23:05:28.228871 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:28.228847 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8ea381_755d_4b7f_8f1f_b57bf5fc8d23.slice/crio-860642aee2fe00d38c2882833be30b124d54b3637950f55a270a87d10d88702a WatchSource:0}: Error finding container 860642aee2fe00d38c2882833be30b124d54b3637950f55a270a87d10d88702a: Status 404 returned error can't find the container with id 860642aee2fe00d38c2882833be30b124d54b3637950f55a270a87d10d88702a Jul 01 23:05:28.242003 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.241937 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nt6sb"] Jul 01 23:05:28.274468 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.274421 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:28.333300 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.333237 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7hsss"] Jul 01 23:05:28.338458 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:28.338429 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07cd1d83_bd84_4673_afbc_dea22ae373e1.slice/crio-970933d9dfc8b5b8a99003f8945fcd146a3e0c71091fb686c667f2af6fdbd0d1 WatchSource:0}: Error finding container 970933d9dfc8b5b8a99003f8945fcd146a3e0c71091fb686c667f2af6fdbd0d1: Status 404 returned error can't find the container with id 970933d9dfc8b5b8a99003f8945fcd146a3e0c71091fb686c667f2af6fdbd0d1 Jul 01 23:05:28.401492 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:28.401452 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ddf54f849-qx9jj"] Jul 01 23:05:28.409793 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:28.409769 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ac79ed_a1fa_4ce2_871a_267fde34f392.slice/crio-25c71c39b82da66b64c10feb4e7dcfb76bf91d6a022bfbefb70a53b821cbd7bb WatchSource:0}: Error finding container 25c71c39b82da66b64c10feb4e7dcfb76bf91d6a022bfbefb70a53b821cbd7bb: Status 404 returned error can't find the container with id 25c71c39b82da66b64c10feb4e7dcfb76bf91d6a022bfbefb70a53b821cbd7bb Jul 01 23:05:29.139589 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.139503 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8msw2" event={"ID":"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23","Type":"ContainerStarted","Data":"860642aee2fe00d38c2882833be30b124d54b3637950f55a270a87d10d88702a"} Jul 01 23:05:29.144592 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.143802 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" event={"ID":"73ac79ed-a1fa-4ce2-871a-267fde34f392","Type":"ContainerStarted","Data":"351282a1df5ac715701e1e02116e3dda2c3fa66b9a40971b6729b3fb76496579"} Jul 01 23:05:29.144592 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.143867 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" event={"ID":"73ac79ed-a1fa-4ce2-871a-267fde34f392","Type":"ContainerStarted","Data":"25c71c39b82da66b64c10feb4e7dcfb76bf91d6a022bfbefb70a53b821cbd7bb"} Jul 01 23:05:29.144592 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.144540 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:29.148930 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.148795 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nt6sb" event={"ID":"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d","Type":"ContainerStarted","Data":"c74a67051c4cd2f0abcf78263ba0334c3bcec6ffd5b96fc1e0dc1f80ae5764dc"} Jul 01 23:05:29.152203 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.152179 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7hsss" event={"ID":"07cd1d83-bd84-4673-afbc-dea22ae373e1","Type":"ContainerStarted","Data":"970933d9dfc8b5b8a99003f8945fcd146a3e0c71091fb686c667f2af6fdbd0d1"} Jul 01 23:05:29.168471 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:29.168425 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" podStartSLOduration=2.168408287 podStartE2EDuration="2.168408287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:29.167211589 +0000 UTC m=+59.389739800" watchObservedRunningTime="2026-07-01 23:05:29.168408287 +0000 UTC m=+59.390936498" Jul 01 23:05:30.145215 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.145182 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs"] Jul 01 23:05:30.147437 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.147415 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:30.153311 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.153243 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 01 23:05:30.153519 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.153500 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-h9j9k\"" Jul 01 23:05:30.164871 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.164852 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs"] Jul 01 23:05:30.225941 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.224957 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8711075-504f-47f5-969c-45b236f501e2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-89rvs\" (UID: \"c8711075-504f-47f5-969c-45b236f501e2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:30.325783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.325744 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8711075-504f-47f5-969c-45b236f501e2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-89rvs\" (UID: \"c8711075-504f-47f5-969c-45b236f501e2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:30.328697 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.328672 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 01 23:05:30.337019 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:30.336987 2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 01 23:05:30.337138 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:30.337054 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8711075-504f-47f5-969c-45b236f501e2-tls-certificates podName:c8711075-504f-47f5-969c-45b236f501e2 nodeName:}" failed. No retries permitted until 2026-07-01 23:05:30.837032966 +0000 UTC m=+61.059561158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c8711075-504f-47f5-969c-45b236f501e2-tls-certificates") pod "prometheus-operator-admission-webhook-7ddc8784d4-89rvs" (UID: "c8711075-504f-47f5-969c-45b236f501e2") : secret "prometheus-operator-admission-webhook-tls" not found Jul 01 23:05:30.930045 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.930009 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8711075-504f-47f5-969c-45b236f501e2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-89rvs\" (UID: \"c8711075-504f-47f5-969c-45b236f501e2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:30.932620 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:30.932579 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c8711075-504f-47f5-969c-45b236f501e2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-89rvs\" (UID: \"c8711075-504f-47f5-969c-45b236f501e2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:31.064695 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:31.064663 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-h9j9k\"" Jul 01 23:05:31.071943 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:31.071914 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:32.086723 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:32.086696 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hbn" Jul 01 23:05:33.840572 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:33.840527 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs"] Jul 01 23:05:33.844703 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:33.844600 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8711075_504f_47f5_969c_45b236f501e2.slice/crio-ce73342362c6b8c4ec407a1833cb1b517b5aacf0a8c3a3300e6e1e3f68851305 WatchSource:0}: Error finding container ce73342362c6b8c4ec407a1833cb1b517b5aacf0a8c3a3300e6e1e3f68851305: Status 404 returned error can't find the container with id ce73342362c6b8c4ec407a1833cb1b517b5aacf0a8c3a3300e6e1e3f68851305 Jul 01 23:05:34.170093 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.170050 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nt6sb" event={"ID":"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d","Type":"ContainerStarted","Data":"b743a40f96b293ae57a3bb76ee9d25233451385044ce77acf35d243539a54e5f"} Jul 01 23:05:34.170291 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.170117 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nt6sb" event={"ID":"d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d","Type":"ContainerStarted","Data":"aeab08c62cb0a45964405662296c132dc494caadbc42ca5b5c8339368bdd2a81"} Jul 01 23:05:34.170291 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.170156 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:34.171187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.171167 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" event={"ID":"c8711075-504f-47f5-969c-45b236f501e2","Type":"ContainerStarted","Data":"ce73342362c6b8c4ec407a1833cb1b517b5aacf0a8c3a3300e6e1e3f68851305"} Jul 01 23:05:34.172539 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.172519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7hsss" event={"ID":"07cd1d83-bd84-4673-afbc-dea22ae373e1","Type":"ContainerStarted","Data":"5ee0a42ba1cd4b44a0a483fa94810911695284051def6b01bdd21c260c22304f"} Jul 01 23:05:34.173749 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.173731 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8msw2" event={"ID":"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23","Type":"ContainerStarted","Data":"1d2eaa76c40e78fcd7078949c1d899ad354c4dc680c78873cb09f033836a7e6c"} Jul 01 23:05:34.190678 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.190636 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nt6sb" podStartSLOduration=1.745706557 podStartE2EDuration="7.190601746s" totalImagesPullingTime="5.444895189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:34.189851792 +0000 UTC m=+64.412380003" watchObservedRunningTime="2026-07-01 23:05:34.190601746 +0000 UTC m=+64.413129993" Jul 01 23:05:34.209973 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:34.209931 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7hsss" podStartSLOduration=1.8309289629999999 podStartE2EDuration="7.20991892s" totalImagesPullingTime="5.378989957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:34.209710742 +0000 UTC m=+64.432238948" watchObservedRunningTime="2026-07-01 23:05:34.20991892 +0000 UTC m=+64.432447122" Jul 01 23:05:35.178149 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:35.178124 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8msw2" event={"ID":"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23","Type":"ContainerStarted","Data":"bcd0b44c1b195513d63ae30fe132f5cd3c4df8a88842b1724f7f4db5cadb57db"} Jul 01 23:05:35.179592 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:35.179567 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" event={"ID":"c8711075-504f-47f5-969c-45b236f501e2","Type":"ContainerStarted","Data":"3fec6f93f224a41f8f22e905a337c27155ab7b3141b8be45d35242c05daeebe7"} Jul 01 23:05:35.180051 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:35.180032 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:35.180981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:35.180956 2569 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-7ddc8784d4-89rvs container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.10:8443/healthz\": dial tcp 10.134.0.10:8443: connect: connection refused" start-of-body= Jul 01 23:05:35.181074 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:35.181000 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" podUID="c8711075-504f-47f5-969c-45b236f501e2" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.134.0.10:8443/healthz\": dial tcp 10.134.0.10:8443: connect: connection refused" Jul 01 23:05:35.195569 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:35.195428 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" podStartSLOduration=3.921741113 podStartE2EDuration="5.195411854s" totalImagesPullingTime="1.273670741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:35.19496374 +0000 UTC m=+65.417491974" watchObservedRunningTime="2026-07-01 23:05:35.195411854 +0000 UTC m=+65.417940064" Jul 01 23:05:36.074269 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.074228 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:36.077818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.077795 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 23:05:36.086821 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.086773 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afd99605-6853-41d1-8b9f-a4aa46695472-original-pull-secret\") pod \"global-pull-secret-syncer-xjqgl\" (UID: \"afd99605-6853-41d1-8b9f-a4aa46695472\") " pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:36.174935 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.174895 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:36.175102 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.174964 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:36.179928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.179897 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 23:05:36.180592 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.180569 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 23:05:36.189958 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.189922 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 23:05:36.190113 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.190090 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31c226b-95f7-4a3f-9b94-476f82172154-metrics-certs\") pod \"network-metrics-daemon-ghqjk\" (UID: \"f31c226b-95f7-4a3f-9b94-476f82172154\") " pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:36.192303 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.192271 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-89rvs" Jul 01 23:05:36.200448 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.200428 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7klzx\" (UniqueName: \"kubernetes.io/projected/2ffdec5f-ae85-4322-973f-bcf570740a6f-kube-api-access-7klzx\") pod \"network-check-target-4pz9m\" (UID: \"2ffdec5f-ae85-4322-973f-bcf570740a6f\") " pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:36.331748 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.331663 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-96nlh\"" Jul 01 23:05:36.339119 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.339096 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:36.341357 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.341338 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjqgl" Jul 01 23:05:36.346086 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.346067 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6x5m9\"" Jul 01 23:05:36.346086 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:36.346078 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghqjk" Jul 01 23:05:37.224291 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.223558 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-hc8rs"] Jul 01 23:05:37.233132 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.233057 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.238813 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.238079 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-hc8rs"] Jul 01 23:05:37.238813 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.238457 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 23:05:37.238813 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.238723 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 23:05:37.239032 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.238953 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 23:05:37.240510 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.239183 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 23:05:37.240510 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.240050 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7kbvd\"" Jul 01 23:05:37.240510 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.240105 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 23:05:37.319888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.319814 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjqgl"] Jul 01 23:05:37.324454 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:37.324410 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd99605_6853_41d1_8b9f_a4aa46695472.slice/crio-6910329648f3acd204a8faccb3c5a707bf9a613f29b38d5cdee37e8e3853495a WatchSource:0}: Error finding container 6910329648f3acd204a8faccb3c5a707bf9a613f29b38d5cdee37e8e3853495a: Status 404 returned error can't find the container with id 6910329648f3acd204a8faccb3c5a707bf9a613f29b38d5cdee37e8e3853495a Jul 01 23:05:37.333389 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.333360 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4pz9m"] Jul 01 23:05:37.336009 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:37.335984 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ffdec5f_ae85_4322_973f_bcf570740a6f.slice/crio-16159d91fb6daefd2757ce0308ac30955fba4571d30999cf196140c6e40d8bf5 WatchSource:0}: Error finding container 16159d91fb6daefd2757ce0308ac30955fba4571d30999cf196140c6e40d8bf5: Status 404 returned error can't find the container with id 16159d91fb6daefd2757ce0308ac30955fba4571d30999cf196140c6e40d8bf5 Jul 01 23:05:37.350132 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.350111 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ghqjk"] Jul 01 23:05:37.353054 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:37.352987 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31c226b_95f7_4a3f_9b94_476f82172154.slice/crio-920246c4c032bfa6e116ec1e9b147e755642dcc40dc36f67b3808a963adb8672 WatchSource:0}: Error finding container 920246c4c032bfa6e116ec1e9b147e755642dcc40dc36f67b3808a963adb8672: Status 404 returned error can't find the container with id 920246c4c032bfa6e116ec1e9b147e755642dcc40dc36f67b3808a963adb8672 Jul 01 23:05:37.382900 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.382871 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.383009 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.382915 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.383009 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.382936 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/236ebef7-24b0-45ad-8b90-edf250868228-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.383078 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.383009 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcn56\" (UniqueName: \"kubernetes.io/projected/236ebef7-24b0-45ad-8b90-edf250868228-kube-api-access-vcn56\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.484268 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.484189 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.484268 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.484227 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.484268 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.484247 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/236ebef7-24b0-45ad-8b90-edf250868228-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.484442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.484273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcn56\" (UniqueName: \"kubernetes.io/projected/236ebef7-24b0-45ad-8b90-edf250868228-kube-api-access-vcn56\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.484442 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:37.484339 2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 01 23:05:37.484442 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:05:37.484409 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-tls podName:236ebef7-24b0-45ad-8b90-edf250868228 nodeName:}" failed. No retries permitted until 2026-07-01 23:05:37.984392682 +0000 UTC m=+68.206920882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-hc8rs" (UID: "236ebef7-24b0-45ad-8b90-edf250868228") : secret "prometheus-operator-tls" not found Jul 01 23:05:37.485024 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.485008 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/236ebef7-24b0-45ad-8b90-edf250868228-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.486522 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.486504 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.494914 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.494891 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcn56\" (UniqueName: \"kubernetes.io/projected/236ebef7-24b0-45ad-8b90-edf250868228-kube-api-access-vcn56\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.987774 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.987742 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:37.990962 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:37.990902 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/236ebef7-24b0-45ad-8b90-edf250868228-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-hc8rs\" (UID: \"236ebef7-24b0-45ad-8b90-edf250868228\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:38.157841 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.157803 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" Jul 01 23:05:38.195965 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.195925 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ghqjk" event={"ID":"f31c226b-95f7-4a3f-9b94-476f82172154","Type":"ContainerStarted","Data":"920246c4c032bfa6e116ec1e9b147e755642dcc40dc36f67b3808a963adb8672"} Jul 01 23:05:38.197685 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.197591 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4pz9m" event={"ID":"2ffdec5f-ae85-4322-973f-bcf570740a6f","Type":"ContainerStarted","Data":"16159d91fb6daefd2757ce0308ac30955fba4571d30999cf196140c6e40d8bf5"} Jul 01 23:05:38.201969 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.201324 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8msw2" event={"ID":"bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23","Type":"ContainerStarted","Data":"5755bb6d62bd3d036867558d880f21076aae8841459fce1222ad16993ee43d23"} Jul 01 23:05:38.203209 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.203154 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjqgl" event={"ID":"afd99605-6853-41d1-8b9f-a4aa46695472","Type":"ContainerStarted","Data":"6910329648f3acd204a8faccb3c5a707bf9a613f29b38d5cdee37e8e3853495a"} Jul 01 23:05:38.235525 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.234968 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8msw2" podStartSLOduration=2.402785464 podStartE2EDuration="11.234947171s" totalImagesPullingTime="8.832161707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:05:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:38.234692294 +0000 UTC m=+68.457220529" watchObservedRunningTime="2026-07-01 23:05:38.234947171 +0000 UTC m=+68.457475381" Jul 01 23:05:38.438100 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:38.438023 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-hc8rs"] Jul 01 23:05:38.441552 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:38.441412 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236ebef7_24b0_45ad_8b90_edf250868228.slice/crio-83012878e242830d351465338011b9a11b59f3c53e400e1028cb3f11a7f2f74a WatchSource:0}: Error finding container 83012878e242830d351465338011b9a11b59f3c53e400e1028cb3f11a7f2f74a: Status 404 returned error can't find the container with id 83012878e242830d351465338011b9a11b59f3c53e400e1028cb3f11a7f2f74a Jul 01 23:05:39.211378 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:39.211335 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" event={"ID":"236ebef7-24b0-45ad-8b90-edf250868228","Type":"ContainerStarted","Data":"83012878e242830d351465338011b9a11b59f3c53e400e1028cb3f11a7f2f74a"} Jul 01 23:05:39.214138 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:39.214104 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ghqjk" event={"ID":"f31c226b-95f7-4a3f-9b94-476f82172154","Type":"ContainerStarted","Data":"036ac15378823bb6e72d1cfbbc950fb7d06b6f0f2c68e1356cb1d75a3ad789ec"} Jul 01 23:05:39.214282 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:39.214163 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ghqjk" event={"ID":"f31c226b-95f7-4a3f-9b94-476f82172154","Type":"ContainerStarted","Data":"27921aca7ecda258acd0381c29563b64aa7047863914c7ffd7a5b7cab0f37a79"} Jul 01 23:05:39.232513 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:39.232449 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ghqjk" podStartSLOduration=67.270728843 podStartE2EDuration="1m8.232430713s" totalImagesPullingTime="961.70187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:39.230645721 +0000 UTC m=+69.453173945" watchObservedRunningTime="2026-07-01 23:05:39.232430713 +0000 UTC m=+69.454958921" Jul 01 23:05:43.234728 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.234695 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4pz9m" event={"ID":"2ffdec5f-ae85-4322-973f-bcf570740a6f","Type":"ContainerStarted","Data":"b50d6c82d07dab412b7224c553bed35133df3f54e85cd76b8f417092c1b940c4"} Jul 01 23:05:43.235123 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.234777 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:05:43.236148 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.236127 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjqgl" event={"ID":"afd99605-6853-41d1-8b9f-a4aa46695472","Type":"ContainerStarted","Data":"133fba34282686376a84a393175a607f49f0ddcf3f2241b4e8ed86f46314354f"} Jul 01 23:05:43.237877 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.237850 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" event={"ID":"236ebef7-24b0-45ad-8b90-edf250868228","Type":"ContainerStarted","Data":"49b384cd74d5dc8a0c173fa303d958abe1e8b23bd8cbe296fe72c98edba9c13c"} Jul 01 23:05:43.237993 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.237886 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" event={"ID":"236ebef7-24b0-45ad-8b90-edf250868228","Type":"ContainerStarted","Data":"540c75ca6c9cf56eae46cbfd702a28690ecce83bde5946f21bbae5a8a41f45ae"} Jul 01 23:05:43.269888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.269838 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4pz9m" podStartSLOduration=67.104632257 podStartE2EDuration="1m12.269822764s" totalImagesPullingTime="5.165190507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:43.251348352 +0000 UTC m=+73.473876561" watchObservedRunningTime="2026-07-01 23:05:43.269822764 +0000 UTC m=+73.492350977" Jul 01 23:05:43.290921 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.290872 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xjqgl" podStartSLOduration=67.110769046 podStartE2EDuration="1m12.290857754s" totalImagesPullingTime="5.180088708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:43.290217019 +0000 UTC m=+73.512745257" watchObservedRunningTime="2026-07-01 23:05:43.290857754 +0000 UTC m=+73.513385966" Jul 01 23:05:43.291289 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:43.291264 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-hc8rs" podStartSLOduration=2.242677264 podStartE2EDuration="6.291257079s" totalImagesPullingTime="4.048579815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:43.271765784 +0000 UTC m=+73.494294011" watchObservedRunningTime="2026-07-01 23:05:43.291257079 +0000 UTC m=+73.513785288" Jul 01 23:05:44.181594 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:44.181566 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nt6sb" Jul 01 23:05:45.605657 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.605626 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv"] Jul 01 23:05:45.607655 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.607639 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.609656 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.609632 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 01 23:05:45.609837 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.609816 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 01 23:05:45.609986 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.609971 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wn79j\"" Jul 01 23:05:45.620720 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.620697 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv"] Jul 01 23:05:45.642984 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.642950 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-8c6fh"] Jul 01 23:05:45.644848 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.644835 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.647767 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.647748 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 01 23:05:45.648146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.648121 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 01 23:05:45.648242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.648188 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 01 23:05:45.648576 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.648562 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sbhj9\"" Jul 01 23:05:45.658138 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.658120 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-8c6fh"] Jul 01 23:05:45.671010 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.670988 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wsxv9"] Jul 01 23:05:45.672830 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.672816 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.675188 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.675108 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 23:05:45.675188 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.675171 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 23:05:45.675344 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.675306 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kbqh6\"" Jul 01 23:05:45.675416 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.675399 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 23:05:45.743913 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.743883 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.744048 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.743920 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrz98\" (UniqueName: \"kubernetes.io/projected/8884db4b-140d-4ee0-ba84-4ae339700754-kube-api-access-wrz98\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.744048 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.743948 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ecf1b55-0754-4c71-84b7-36b67f3d077c-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.744048 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.743998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.744048 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.744041 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ecf1b55-0754-4c71-84b7-36b67f3d077c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.744177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.744077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ecf1b55-0754-4c71-84b7-36b67f3d077c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.744177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.744121 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8884db4b-140d-4ee0-ba84-4ae339700754-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.744177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.744144 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8884db4b-140d-4ee0-ba84-4ae339700754-metrics-client-ca\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.744177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.744161 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.744300 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.744188 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bd2n\" (UniqueName: \"kubernetes.io/projected/3ecf1b55-0754-4c71-84b7-36b67f3d077c-kube-api-access-2bd2n\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.845400 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845361 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrz98\" (UniqueName: \"kubernetes.io/projected/8884db4b-140d-4ee0-ba84-4ae339700754-kube-api-access-wrz98\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.845400 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845402 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ecf1b55-0754-4c71-84b7-36b67f3d077c-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.845651 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845423 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-wtmp\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.845651 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.845651 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845559 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msgkq\" (UniqueName: \"kubernetes.io/projected/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-kube-api-access-msgkq\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.845818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ecf1b55-0754-4c71-84b7-36b67f3d077c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.845818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.845818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845800 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-root\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.845963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ecf1b55-0754-4c71-84b7-36b67f3d077c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.845963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845897 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8884db4b-140d-4ee0-ba84-4ae339700754-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.845963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845924 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8884db4b-140d-4ee0-ba84-4ae339700754-metrics-client-ca\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.845963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.846152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.845978 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-metrics-client-ca\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.846152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bd2n\" (UniqueName: \"kubernetes.io/projected/3ecf1b55-0754-4c71-84b7-36b67f3d077c-kube-api-access-2bd2n\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.846152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846054 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-tls\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.846152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846095 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-textfile\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.846152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846129 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-sys\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.846387 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846156 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.846387 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846188 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.846746 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846718 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ecf1b55-0754-4c71-84b7-36b67f3d077c-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.846847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846812 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8884db4b-140d-4ee0-ba84-4ae339700754-metrics-client-ca\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.846847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.846823 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.847085 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.847062 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8884db4b-140d-4ee0-ba84-4ae339700754-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.848170 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.848146 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.848320 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.848298 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ecf1b55-0754-4c71-84b7-36b67f3d077c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.848450 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.848432 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ecf1b55-0754-4c71-84b7-36b67f3d077c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.848583 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.848551 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8884db4b-140d-4ee0-ba84-4ae339700754-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.859993 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.859942 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrz98\" (UniqueName: \"kubernetes.io/projected/8884db4b-140d-4ee0-ba84-4ae339700754-kube-api-access-wrz98\") pod \"kube-state-metrics-845b954455-8c6fh\" (UID: \"8884db4b-140d-4ee0-ba84-4ae339700754\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.862669 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.862648 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bd2n\" (UniqueName: \"kubernetes.io/projected/3ecf1b55-0754-4c71-84b7-36b67f3d077c-kube-api-access-2bd2n\") pod \"openshift-state-metrics-57f99c8b49-g2rqv\" (UID: \"3ecf1b55-0754-4c71-84b7-36b67f3d077c\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.916543 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.916519 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" Jul 01 23:05:45.946827 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946801 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-root\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.946990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-metrics-client-ca\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.946990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946865 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-tls\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.946990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946909 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-root\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.946990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-textfile\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.946990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946977 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-sys\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.946995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-wtmp\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947053 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msgkq\" (UniqueName: \"kubernetes.io/projected/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-kube-api-access-msgkq\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947072 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-sys\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947174 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947205 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-wtmp\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947531 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947391 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-textfile\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947531 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947476 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-metrics-client-ca\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.947818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.947802 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-accelerators-collector-config\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.949113 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.949094 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-tls\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.949692 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.949676 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.952673 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.952657 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" Jul 01 23:05:45.961965 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.961893 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msgkq\" (UniqueName: \"kubernetes.io/projected/3fbee254-bf29-46fa-b44a-a104ee5e9ab2-kube-api-access-msgkq\") pod \"node-exporter-wsxv9\" (UID: \"3fbee254-bf29-46fa-b44a-a104ee5e9ab2\") " pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.982000 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:45.981942 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wsxv9" Jul 01 23:05:45.993045 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:45.993017 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbee254_bf29_46fa_b44a_a104ee5e9ab2.slice/crio-c7ce67a8ebbf37df3c9793302996b95960c9cc8e767f07edd458232925507af1 WatchSource:0}: Error finding container c7ce67a8ebbf37df3c9793302996b95960c9cc8e767f07edd458232925507af1: Status 404 returned error can't find the container with id c7ce67a8ebbf37df3c9793302996b95960c9cc8e767f07edd458232925507af1 Jul 01 23:05:46.051228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.051191 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv"] Jul 01 23:05:46.056974 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:46.056942 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecf1b55_0754_4c71_84b7_36b67f3d077c.slice/crio-82e35e19101dc2e47bd7d22b4a98e38a18d08049dd5a703843bc8e5f14444756 WatchSource:0}: Error finding container 82e35e19101dc2e47bd7d22b4a98e38a18d08049dd5a703843bc8e5f14444756: Status 404 returned error can't find the container with id 82e35e19101dc2e47bd7d22b4a98e38a18d08049dd5a703843bc8e5f14444756 Jul 01 23:05:46.095307 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.095283 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-8c6fh"] Jul 01 23:05:46.097956 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:46.097933 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8884db4b_140d_4ee0_ba84_4ae339700754.slice/crio-2f97cb11967c679fcdce9b3101e442655555a351102f50d0811e26b65c25f57e WatchSource:0}: Error finding container 2f97cb11967c679fcdce9b3101e442655555a351102f50d0811e26b65c25f57e: Status 404 returned error can't find the container with id 2f97cb11967c679fcdce9b3101e442655555a351102f50d0811e26b65c25f57e Jul 01 23:05:46.248971 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.248935 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsxv9" event={"ID":"3fbee254-bf29-46fa-b44a-a104ee5e9ab2","Type":"ContainerStarted","Data":"c7ce67a8ebbf37df3c9793302996b95960c9cc8e767f07edd458232925507af1"} Jul 01 23:05:46.250028 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.250005 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" event={"ID":"8884db4b-140d-4ee0-ba84-4ae339700754","Type":"ContainerStarted","Data":"2f97cb11967c679fcdce9b3101e442655555a351102f50d0811e26b65c25f57e"} Jul 01 23:05:46.251795 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.251775 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" event={"ID":"3ecf1b55-0754-4c71-84b7-36b67f3d077c","Type":"ContainerStarted","Data":"08893b0568247e6f76748bd59cd38a561fdb0539ffead899331928d8bd68d865"} Jul 01 23:05:46.251881 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.251814 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" event={"ID":"3ecf1b55-0754-4c71-84b7-36b67f3d077c","Type":"ContainerStarted","Data":"712fcc5261282dfba98fdb42b8ecefb56af45cbde2209dc29be618b2a03bc7f1"} Jul 01 23:05:46.251881 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.251839 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" event={"ID":"3ecf1b55-0754-4c71-84b7-36b67f3d077c","Type":"ContainerStarted","Data":"82e35e19101dc2e47bd7d22b4a98e38a18d08049dd5a703843bc8e5f14444756"} Jul 01 23:05:46.678870 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.678650 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:05:46.682208 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.682133 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.684839 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.684691 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 01 23:05:46.684839 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.684745 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 01 23:05:46.685151 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685128 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 01 23:05:46.685285 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685241 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 01 23:05:46.685523 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685505 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jg4qp\"" Jul 01 23:05:46.685601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685542 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 01 23:05:46.685601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685577 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 01 23:05:46.685749 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685725 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 01 23:05:46.685928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685836 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 01 23:05:46.685928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.685918 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 01 23:05:46.709343 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.709263 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:05:46.853759 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853714 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.853977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853780 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.853977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.853977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853912 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-out\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.853977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853945 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-web-config\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.853977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853971 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.853995 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.854023 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-volume\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.854053 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.854078 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.854103 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.854153 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-428vf\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-kube-api-access-428vf\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.854228 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.854177 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955117 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955175 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955210 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955241 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-out\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955275 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-web-config\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955327 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-volume\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955377 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955400 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955423 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955471 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-428vf\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-kube-api-access-428vf\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.955867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.955497 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.956825 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.956226 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.956825 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.956509 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.958395 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.958342 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.960410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.960386 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.960878 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.960854 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.962305 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.962279 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-web-config\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.962406 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.962376 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-out\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.963007 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.962955 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-volume\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.963007 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.962956 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.963118 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.963063 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.963797 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.963777 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.964973 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.964952 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.968458 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.968434 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-428vf\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-kube-api-access-428vf\") pod \"alertmanager-main-0\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:46.995054 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:46.994861 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:05:47.258846 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.258748 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsxv9" event={"ID":"3fbee254-bf29-46fa-b44a-a104ee5e9ab2","Type":"ContainerStarted","Data":"ecf4ba3c674e0ef4d1182f795ab079b97a00061f97162f043d2d54f7dfe54a3e"} Jul 01 23:05:47.674575 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.674441 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:05:47.683946 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:47.683919 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5efcfbd_532f_4e20_a5dd_4a09ec5f3eab.slice/crio-15e4be2937475f619694639cfb6461df1669922ebf12ad9c78e115dcc2bead13 WatchSource:0}: Error finding container 15e4be2937475f619694639cfb6461df1669922ebf12ad9c78e115dcc2bead13: Status 404 returned error can't find the container with id 15e4be2937475f619694639cfb6461df1669922ebf12ad9c78e115dcc2bead13 Jul 01 23:05:47.853816 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.853785 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z"] Jul 01 23:05:47.856250 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.856228 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.858722 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.858697 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 23:05:47.858868 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.858845 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-76jr4\"" Jul 01 23:05:47.859070 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.859045 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 23:05:47.859179 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.859047 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 23:05:47.859179 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.859049 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 23:05:47.859294 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.859183 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 23:05:47.859294 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.859047 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5u4jbol08tlfg\"" Jul 01 23:05:47.867143 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867116 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867232 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867165 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867309 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867246 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867309 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867293 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-grpc-tls\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867338 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867363 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29399438-37e1-46f3-b3d6-563eb4d118d7-metrics-client-ca\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867508 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867408 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b82h\" (UniqueName: \"kubernetes.io/projected/29399438-37e1-46f3-b3d6-563eb4d118d7-kube-api-access-2b82h\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.867508 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.867441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-tls\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.871294 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.871275 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z"] Jul 01 23:05:47.968204 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968159 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968390 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968390 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968341 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968390 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968365 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-grpc-tls\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968403 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29399438-37e1-46f3-b3d6-563eb4d118d7-metrics-client-ca\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968443 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b82h\" (UniqueName: \"kubernetes.io/projected/29399438-37e1-46f3-b3d6-563eb4d118d7-kube-api-access-2b82h\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.968550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.968477 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-tls\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.969368 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.969340 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29399438-37e1-46f3-b3d6-563eb4d118d7-metrics-client-ca\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.971088 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.971059 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.971301 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.971278 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.971376 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.971338 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.971456 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.971435 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.971661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.971634 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-thanos-querier-tls\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.972019 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.971999 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29399438-37e1-46f3-b3d6-563eb4d118d7-secret-grpc-tls\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:47.978701 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:47.978652 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b82h\" (UniqueName: \"kubernetes.io/projected/29399438-37e1-46f3-b3d6-563eb4d118d7-kube-api-access-2b82h\") pod \"thanos-querier-d9b5b7cd-jfg5z\" (UID: \"29399438-37e1-46f3-b3d6-563eb4d118d7\") " pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:48.170555 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.170464 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:48.267833 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.267790 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" event={"ID":"8884db4b-140d-4ee0-ba84-4ae339700754","Type":"ContainerStarted","Data":"38e7d8eb120588bdf344029a762d725383f3e4c91eb21835b912c1e0a438c1e0"} Jul 01 23:05:48.267964 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.267872 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" event={"ID":"8884db4b-140d-4ee0-ba84-4ae339700754","Type":"ContainerStarted","Data":"ec15987b7c021e154afd3714931de4c68a24f9d43d56942c4bf32dd3bb6eb436"} Jul 01 23:05:48.267964 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.267907 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" event={"ID":"8884db4b-140d-4ee0-ba84-4ae339700754","Type":"ContainerStarted","Data":"43eb8790d744e3c4b2990e84835be79fd1b7d2882462367197bd7735949d9dca"} Jul 01 23:05:48.271477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.271448 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" event={"ID":"3ecf1b55-0754-4c71-84b7-36b67f3d077c","Type":"ContainerStarted","Data":"cbabfb625ef42c38a5e54d62a070a43efe9222777b6a06b1122f8066812bddbf"} Jul 01 23:05:48.273650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.273624 2569 generic.go:356] "Generic (PLEG): container finished" podID="3fbee254-bf29-46fa-b44a-a104ee5e9ab2" containerID="ecf4ba3c674e0ef4d1182f795ab079b97a00061f97162f043d2d54f7dfe54a3e" exitCode=0 Jul 01 23:05:48.273780 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.273711 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsxv9" event={"ID":"3fbee254-bf29-46fa-b44a-a104ee5e9ab2","Type":"ContainerDied","Data":"ecf4ba3c674e0ef4d1182f795ab079b97a00061f97162f043d2d54f7dfe54a3e"} Jul 01 23:05:48.276706 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.276656 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"15e4be2937475f619694639cfb6461df1669922ebf12ad9c78e115dcc2bead13"} Jul 01 23:05:48.288631 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.288564 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-8c6fh" podStartSLOduration=1.8834018019999998 podStartE2EDuration="3.288542198s" totalImagesPullingTime="1.405140396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:48.286960657 +0000 UTC m=+78.509488890" watchObservedRunningTime="2026-07-01 23:05:48.288542198 +0000 UTC m=+78.511070410" Jul 01 23:05:48.329580 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.329522 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-g2rqv" podStartSLOduration=1.977668668 podStartE2EDuration="3.32950395s" totalImagesPullingTime="1.351835282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:48.328812067 +0000 UTC m=+78.551340277" watchObservedRunningTime="2026-07-01 23:05:48.32950395 +0000 UTC m=+78.552032162" Jul 01 23:05:48.334192 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:48.334159 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z"] Jul 01 23:05:48.338482 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:48.338451 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29399438_37e1_46f3_b3d6_563eb4d118d7.slice/crio-50348279e47b031fc36629d922a8e7792a56e1e78a8a98c1716c41beaf77737f WatchSource:0}: Error finding container 50348279e47b031fc36629d922a8e7792a56e1e78a8a98c1716c41beaf77737f: Status 404 returned error can't find the container with id 50348279e47b031fc36629d922a8e7792a56e1e78a8a98c1716c41beaf77737f Jul 01 23:05:49.284645 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:49.284544 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68" exitCode=0 Jul 01 23:05:49.285075 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:49.284642 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68"} Jul 01 23:05:49.286437 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:49.286392 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"50348279e47b031fc36629d922a8e7792a56e1e78a8a98c1716c41beaf77737f"} Jul 01 23:05:49.288916 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:49.288877 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsxv9" event={"ID":"3fbee254-bf29-46fa-b44a-a104ee5e9ab2","Type":"ContainerStarted","Data":"8e5ba754ff90a803cfd7748ea83198b423f03074061aa25660ce70f9ac3c658f"} Jul 01 23:05:49.289030 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:49.288928 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wsxv9" event={"ID":"3fbee254-bf29-46fa-b44a-a104ee5e9ab2","Type":"ContainerStarted","Data":"c0fd1c123a9596069859a2e3c6f35b9968d925419ae5a449edb136dde0bef99e"} Jul 01 23:05:49.347439 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:49.347383 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wsxv9" podStartSLOduration=3.391995181 podStartE2EDuration="4.347364247s" totalImagesPullingTime="879.343929ms" totalInitContainerRuntime="76.025137ms" isStatefulPod=true podCreationTimestamp="2026-07-01 23:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:49.34608551 +0000 UTC m=+79.568613749" watchObservedRunningTime="2026-07-01 23:05:49.347364247 +0000 UTC m=+79.569892460" Jul 01 23:05:50.407635 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.407583 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-pnc95"] Jul 01 23:05:50.411116 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.411089 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.413372 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.413346 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qtb4r\"" Jul 01 23:05:50.413372 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.413367 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 01 23:05:50.420256 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.420202 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-pnc95"] Jul 01 23:05:50.486923 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.486894 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/abc751df-5651-45e3-97b6-b1107ab2dc6c-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-pnc95\" (UID: \"abc751df-5651-45e3-97b6-b1107ab2dc6c\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.487068 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.487001 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cfwj\" (UniqueName: \"kubernetes.io/projected/abc751df-5651-45e3-97b6-b1107ab2dc6c-kube-api-access-6cfwj\") pod \"monitoring-plugin-cd8656b8-pnc95\" (UID: \"abc751df-5651-45e3-97b6-b1107ab2dc6c\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.587815 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.587774 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cfwj\" (UniqueName: \"kubernetes.io/projected/abc751df-5651-45e3-97b6-b1107ab2dc6c-kube-api-access-6cfwj\") pod \"monitoring-plugin-cd8656b8-pnc95\" (UID: \"abc751df-5651-45e3-97b6-b1107ab2dc6c\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.587966 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.587843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/abc751df-5651-45e3-97b6-b1107ab2dc6c-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-pnc95\" (UID: \"abc751df-5651-45e3-97b6-b1107ab2dc6c\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.590419 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.590396 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/abc751df-5651-45e3-97b6-b1107ab2dc6c-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-pnc95\" (UID: \"abc751df-5651-45e3-97b6-b1107ab2dc6c\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.598156 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.598132 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cfwj\" (UniqueName: \"kubernetes.io/projected/abc751df-5651-45e3-97b6-b1107ab2dc6c-kube-api-access-6cfwj\") pod \"monitoring-plugin-cd8656b8-pnc95\" (UID: \"abc751df-5651-45e3-97b6-b1107ab2dc6c\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.722640 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.722534 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:50.973453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.973374 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-vnw62"] Jul 01 23:05:50.976750 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.976723 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:05:50.978966 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.978941 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 23:05:50.979124 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.978941 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 23:05:50.979124 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.978947 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-stcjw\"" Jul 01 23:05:50.987679 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.987654 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-vnw62"] Jul 01 23:05:50.992116 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:50.992087 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6ckf\" (UniqueName: \"kubernetes.io/projected/031efe8c-bb8d-489f-97fc-4195683ba594-kube-api-access-n6ckf\") pod \"downloads-5499b6fb8f-vnw62\" (UID: \"031efe8c-bb8d-489f-97fc-4195683ba594\") " pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:05:51.092834 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.092807 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6ckf\" (UniqueName: \"kubernetes.io/projected/031efe8c-bb8d-489f-97fc-4195683ba594-kube-api-access-n6ckf\") pod \"downloads-5499b6fb8f-vnw62\" (UID: \"031efe8c-bb8d-489f-97fc-4195683ba594\") " pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:05:51.101024 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.101001 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6ckf\" (UniqueName: \"kubernetes.io/projected/031efe8c-bb8d-489f-97fc-4195683ba594-kube-api-access-n6ckf\") pod \"downloads-5499b6fb8f-vnw62\" (UID: \"031efe8c-bb8d-489f-97fc-4195683ba594\") " pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:05:51.165375 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.165356 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-ddf54f849-qx9jj" Jul 01 23:05:51.282110 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.281828 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-pnc95"] Jul 01 23:05:51.285403 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:51.285362 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc751df_5651_45e3_97b6_b1107ab2dc6c.slice/crio-bc8288825925c3d7a81158820bae64e90f3d046f4238610ec2dd97d37849f598 WatchSource:0}: Error finding container bc8288825925c3d7a81158820bae64e90f3d046f4238610ec2dd97d37849f598: Status 404 returned error can't find the container with id bc8288825925c3d7a81158820bae64e90f3d046f4238610ec2dd97d37849f598 Jul 01 23:05:51.287913 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.287875 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:05:51.299622 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.299582 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"b361e466f1f1633a9c5142551fef6e73b35dd6dbf75e7ca19e05821836e5de01"} Jul 01 23:05:51.301000 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.300958 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" event={"ID":"abc751df-5651-45e3-97b6-b1107ab2dc6c","Type":"ContainerStarted","Data":"bc8288825925c3d7a81158820bae64e90f3d046f4238610ec2dd97d37849f598"} Jul 01 23:05:51.303858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.303832 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9"} Jul 01 23:05:51.435790 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:51.435763 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-vnw62"] Jul 01 23:05:51.440541 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:51.440518 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031efe8c_bb8d_489f_97fc_4195683ba594.slice/crio-de8f3929db68a076ce46d1968bfbf826e87396dcfa56ef007ddb121621ed81cd WatchSource:0}: Error finding container de8f3929db68a076ce46d1968bfbf826e87396dcfa56ef007ddb121621ed81cd: Status 404 returned error can't find the container with id de8f3929db68a076ce46d1968bfbf826e87396dcfa56ef007ddb121621ed81cd Jul 01 23:05:52.311989 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.311920 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-vnw62" event={"ID":"031efe8c-bb8d-489f-97fc-4195683ba594","Type":"ContainerStarted","Data":"de8f3929db68a076ce46d1968bfbf826e87396dcfa56ef007ddb121621ed81cd"} Jul 01 23:05:52.315429 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.315359 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"a5315d8753b6ebaaaedf6e01d95d145bc48ce8f27988634a89cb0e558ede15c7"} Jul 01 23:05:52.315752 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.315692 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"40af8a3ff87e9c0a29d19e5fbbf00720ad074cd4c1169d2aca9e6bd6d84f71af"} Jul 01 23:05:52.319627 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.319589 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737"} Jul 01 23:05:52.319719 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.319669 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda"} Jul 01 23:05:52.319719 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.319705 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91"} Jul 01 23:05:52.319868 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:52.319745 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1"} Jul 01 23:05:53.330467 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.330434 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"8c3b28ffcc5a94fb7bc5ef05e1b6971b9d7881b24544e7b485164379f9dd18d0"} Jul 01 23:05:53.330931 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.330530 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"195a733c6ab6dbd52e737ca9c99f172380cea65711cda8c1a7e3e685b9302288"} Jul 01 23:05:53.330931 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.330559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" event={"ID":"29399438-37e1-46f3-b3d6-563eb4d118d7","Type":"ContainerStarted","Data":"1accc612c3da7d56659cf301b836858d2c19b8c132692ffd6eb560340cd1855c"} Jul 01 23:05:53.330931 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.330734 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:53.331995 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.331971 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" event={"ID":"abc751df-5651-45e3-97b6-b1107ab2dc6c","Type":"ContainerStarted","Data":"53ea87d4a28c42c1f6a661776d7c56412890878455561a82bbb2cac9339e1814"} Jul 01 23:05:53.332138 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.332081 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:53.335512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.335491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerStarted","Data":"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b"} Jul 01 23:05:53.337385 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.337363 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" Jul 01 23:05:53.354650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.354588 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" podStartSLOduration=2.575830078 podStartE2EDuration="6.354572787s" totalImagesPullingTime="3.778742709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:53.353980497 +0000 UTC m=+83.576508710" watchObservedRunningTime="2026-07-01 23:05:53.354572787 +0000 UTC m=+83.577100997" Jul 01 23:05:53.383050 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.383012 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.447794927 podStartE2EDuration="7.382997339s" totalImagesPullingTime="3.912372967s" totalInitContainerRuntime="22.829445ms" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:53.38047975 +0000 UTC m=+83.603008001" watchObservedRunningTime="2026-07-01 23:05:53.382997339 +0000 UTC m=+83.605525553" Jul 01 23:05:53.399065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:53.399015 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-pnc95" podStartSLOduration=1.805720403 podStartE2EDuration="3.39899703s" totalImagesPullingTime="1.593276627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:05:53.39777155 +0000 UTC m=+83.620299788" watchObservedRunningTime="2026-07-01 23:05:53.39899703 +0000 UTC m=+83.621525242" Jul 01 23:05:59.092366 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.092333 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79b648fd94-d5m9z"] Jul 01 23:05:59.097001 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.096983 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.099763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.099739 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 23:05:59.099763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.099753 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kprtr\"" Jul 01 23:05:59.099908 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.099739 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 23:05:59.099908 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.099740 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 23:05:59.100445 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.100422 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 23:05:59.100563 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.100433 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 23:05:59.105453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.105432 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79b648fd94-d5m9z"] Jul 01 23:05:59.170341 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.170312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-console-config\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.170469 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.170349 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-oauth-config\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.170469 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.170379 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-serving-cert\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.170546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.170464 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-service-ca\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.170546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.170495 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-oauth-serving-cert\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.170546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.170515 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfz86\" (UniqueName: \"kubernetes.io/projected/598d38df-446b-4f14-9492-7474419634d6-kube-api-access-wfz86\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.271579 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.271546 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-oauth-config\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.271741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.271625 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-serving-cert\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.271741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.271666 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-service-ca\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.271741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.271687 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-oauth-serving-cert\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.271741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.271712 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfz86\" (UniqueName: \"kubernetes.io/projected/598d38df-446b-4f14-9492-7474419634d6-kube-api-access-wfz86\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.271983 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.271791 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-console-config\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.272542 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.272516 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-console-config\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.272542 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.272532 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-service-ca\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.272888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.272851 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-oauth-serving-cert\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.274251 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.274230 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-oauth-config\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.274251 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.274244 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-serving-cert\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.281417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.281392 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfz86\" (UniqueName: \"kubernetes.io/projected/598d38df-446b-4f14-9492-7474419634d6-kube-api-access-wfz86\") pod \"console-79b648fd94-d5m9z\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.346468 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.346417 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d9b5b7cd-jfg5z" Jul 01 23:05:59.407599 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.407572 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:05:59.552765 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:05:59.552732 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79b648fd94-d5m9z"] Jul 01 23:05:59.556229 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:05:59.556204 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598d38df_446b_4f14_9492_7474419634d6.slice/crio-49cd04f228aa8cdaae3971e37e09b41293ba02b487151156eb938c7ee5aeab0b WatchSource:0}: Error finding container 49cd04f228aa8cdaae3971e37e09b41293ba02b487151156eb938c7ee5aeab0b: Status 404 returned error can't find the container with id 49cd04f228aa8cdaae3971e37e09b41293ba02b487151156eb938c7ee5aeab0b Jul 01 23:06:00.371217 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:00.371175 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b648fd94-d5m9z" event={"ID":"598d38df-446b-4f14-9492-7474419634d6","Type":"ContainerStarted","Data":"49cd04f228aa8cdaae3971e37e09b41293ba02b487151156eb938c7ee5aeab0b"} Jul 01 23:06:04.392437 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:04.392394 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b648fd94-d5m9z" event={"ID":"598d38df-446b-4f14-9492-7474419634d6","Type":"ContainerStarted","Data":"bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610"} Jul 01 23:06:04.414004 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:04.413938 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79b648fd94-d5m9z" podStartSLOduration=1.5403108429999999 podStartE2EDuration="5.413916931s" totalImagesPullingTime="3.873606088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:06:04.411298509 +0000 UTC m=+94.633826748" watchObservedRunningTime="2026-07-01 23:06:04.413916931 +0000 UTC m=+94.636445142" Jul 01 23:06:06.276591 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.276556 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8696cd744d-jn8zj"] Jul 01 23:06:06.279815 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.279794 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.288783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.288758 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 23:06:06.292701 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.292677 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8696cd744d-jn8zj"] Jul 01 23:06:06.340403 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340370 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-console-config\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.340568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340417 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-oauth-serving-cert\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.340568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340439 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-serving-cert\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.340568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340523 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-service-ca\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.340568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340559 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-oauth-config\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.340766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340588 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-trusted-ca-bundle\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.340766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.340646 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdslx\" (UniqueName: \"kubernetes.io/projected/8c891f31-5850-4014-b72c-e6a34befa288-kube-api-access-wdslx\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441458 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-console-config\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-oauth-serving-cert\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441511 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-serving-cert\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441539 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-service-ca\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441563 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-oauth-config\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-trusted-ca-bundle\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.441660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.441629 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdslx\" (UniqueName: \"kubernetes.io/projected/8c891f31-5850-4014-b72c-e6a34befa288-kube-api-access-wdslx\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.442254 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.442224 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-console-config\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.442379 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.442268 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-oauth-serving-cert\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.442652 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.442593 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-service-ca\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.442910 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.442886 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-trusted-ca-bundle\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.444275 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.444257 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-serving-cert\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.444379 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.444359 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-oauth-config\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.450977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.450958 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdslx\" (UniqueName: \"kubernetes.io/projected/8c891f31-5850-4014-b72c-e6a34befa288-kube-api-access-wdslx\") pod \"console-8696cd744d-jn8zj\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:06.592296 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:06.592261 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:09.407956 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:09.407917 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:06:09.408366 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:09.407982 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:06:09.413739 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:09.413709 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:06:09.418157 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:09.418137 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:06:12.069974 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.069946 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8696cd744d-jn8zj"] Jul 01 23:06:12.072474 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:06:12.072448 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c891f31_5850_4014_b72c_e6a34befa288.slice/crio-2ee1e82496788c12004acb7c45306ddd4c2031543f0577132f9d577c73c7b643 WatchSource:0}: Error finding container 2ee1e82496788c12004acb7c45306ddd4c2031543f0577132f9d577c73c7b643: Status 404 returned error can't find the container with id 2ee1e82496788c12004acb7c45306ddd4c2031543f0577132f9d577c73c7b643 Jul 01 23:06:12.427977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.427948 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696cd744d-jn8zj" event={"ID":"8c891f31-5850-4014-b72c-e6a34befa288","Type":"ContainerStarted","Data":"a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676"} Jul 01 23:06:12.428098 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.428008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696cd744d-jn8zj" event={"ID":"8c891f31-5850-4014-b72c-e6a34befa288","Type":"ContainerStarted","Data":"2ee1e82496788c12004acb7c45306ddd4c2031543f0577132f9d577c73c7b643"} Jul 01 23:06:12.429483 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.429459 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-vnw62" event={"ID":"031efe8c-bb8d-489f-97fc-4195683ba594","Type":"ContainerStarted","Data":"b641bd3acedf3f63bed8e4dff1da91a8741d83027c73e210e1f7769a46cf2756"} Jul 01 23:06:12.429584 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.429566 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:06:12.431194 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.431172 2569 patch_prober.go:28] interesting pod/downloads-5499b6fb8f-vnw62 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.17:8080/\": dial tcp 10.134.0.17:8080: connect: connection refused" start-of-body= Jul 01 23:06:12.431249 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.431220 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5499b6fb8f-vnw62" podUID="031efe8c-bb8d-489f-97fc-4195683ba594" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.17:8080/\": dial tcp 10.134.0.17:8080: connect: connection refused" Jul 01 23:06:12.447773 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.447622 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8696cd744d-jn8zj" podStartSLOduration=6.447589769 podStartE2EDuration="6.447589769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:06:12.446513856 +0000 UTC m=+102.669042092" watchObservedRunningTime="2026-07-01 23:06:12.447589769 +0000 UTC m=+102.670117979" Jul 01 23:06:12.463193 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:12.463144 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-vnw62" podStartSLOduration=1.5484784120000001 podStartE2EDuration="22.463128758s" totalImagesPullingTime="20.914650346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:06:12.461636713 +0000 UTC m=+102.684164952" watchObservedRunningTime="2026-07-01 23:06:12.463128758 +0000 UTC m=+102.685656967" Jul 01 23:06:13.450180 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:13.450150 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-vnw62" Jul 01 23:06:14.244420 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:14.244380 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4pz9m" Jul 01 23:06:16.592363 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:16.592330 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:16.592828 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:16.592401 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:16.597994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:16.597969 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:17.455854 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:17.455821 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:06:17.503350 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:17.503315 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79b648fd94-d5m9z"] Jul 01 23:06:42.529500 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.529436 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79b648fd94-d5m9z" podUID="598d38df-446b-4f14-9492-7474419634d6" containerName="console" containerID="cri-o://bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610" gracePeriod=15 Jul 01 23:06:42.804656 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.804563 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:06:42.841322 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841291 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-service-ca\") pod \"598d38df-446b-4f14-9492-7474419634d6\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " Jul 01 23:06:42.841505 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841335 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-console-config\") pod \"598d38df-446b-4f14-9492-7474419634d6\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " Jul 01 23:06:42.841505 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841368 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-oauth-serving-cert\") pod \"598d38df-446b-4f14-9492-7474419634d6\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " Jul 01 23:06:42.841505 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841470 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-serving-cert\") pod \"598d38df-446b-4f14-9492-7474419634d6\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " Jul 01 23:06:42.841709 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841529 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-oauth-config\") pod \"598d38df-446b-4f14-9492-7474419634d6\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " Jul 01 23:06:42.841709 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841554 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfz86\" (UniqueName: \"kubernetes.io/projected/598d38df-446b-4f14-9492-7474419634d6-kube-api-access-wfz86\") pod \"598d38df-446b-4f14-9492-7474419634d6\" (UID: \"598d38df-446b-4f14-9492-7474419634d6\") " Jul 01 23:06:42.841817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841781 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-console-config" (OuterVolumeSpecName: "console-config") pod "598d38df-446b-4f14-9492-7474419634d6" (UID: "598d38df-446b-4f14-9492-7474419634d6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:06:42.841817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841790 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "598d38df-446b-4f14-9492-7474419634d6" (UID: "598d38df-446b-4f14-9492-7474419634d6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:06:42.841817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841799 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-service-ca" (OuterVolumeSpecName: "service-ca") pod "598d38df-446b-4f14-9492-7474419634d6" (UID: "598d38df-446b-4f14-9492-7474419634d6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:06:42.841952 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841908 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-service-ca\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:06:42.841952 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841926 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-console-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:06:42.841952 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.841939 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/598d38df-446b-4f14-9492-7474419634d6-oauth-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:06:42.843775 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.843702 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "598d38df-446b-4f14-9492-7474419634d6" (UID: "598d38df-446b-4f14-9492-7474419634d6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:06:42.843775 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.843735 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598d38df-446b-4f14-9492-7474419634d6-kube-api-access-wfz86" (OuterVolumeSpecName: "kube-api-access-wfz86") pod "598d38df-446b-4f14-9492-7474419634d6" (UID: "598d38df-446b-4f14-9492-7474419634d6"). InnerVolumeSpecName "kube-api-access-wfz86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:06:42.843775 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.843757 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "598d38df-446b-4f14-9492-7474419634d6" (UID: "598d38df-446b-4f14-9492-7474419634d6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:06:42.942471 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.942447 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:06:42.942471 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.942471 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/598d38df-446b-4f14-9492-7474419634d6-console-oauth-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:06:42.942638 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:42.942481 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfz86\" (UniqueName: \"kubernetes.io/projected/598d38df-446b-4f14-9492-7474419634d6-kube-api-access-wfz86\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:06:43.550543 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.550510 2569 generic.go:356] "Generic (PLEG): container finished" podID="598d38df-446b-4f14-9492-7474419634d6" containerID="bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610" exitCode=0 Jul 01 23:06:43.550950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.550558 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79b648fd94-d5m9z" Jul 01 23:06:43.550950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.550590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b648fd94-d5m9z" event={"ID":"598d38df-446b-4f14-9492-7474419634d6","Type":"ContainerDied","Data":"bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610"} Jul 01 23:06:43.550950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.550655 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79b648fd94-d5m9z" event={"ID":"598d38df-446b-4f14-9492-7474419634d6","Type":"ContainerDied","Data":"49cd04f228aa8cdaae3971e37e09b41293ba02b487151156eb938c7ee5aeab0b"} Jul 01 23:06:43.550950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.550675 2569 scope.go:117] "RemoveContainer" containerID="bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610" Jul 01 23:06:43.568885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.568858 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79b648fd94-d5m9z"] Jul 01 23:06:43.573529 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.573506 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79b648fd94-d5m9z"] Jul 01 23:06:43.574064 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.574048 2569 scope.go:117] "RemoveContainer" containerID="bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610" Jul 01 23:06:43.574323 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:06:43.574303 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610\": container with ID starting with bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610 not found: ID does not exist" containerID="bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610" Jul 01 23:06:43.574384 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:43.574332 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610"} err="failed to get container status \"bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610\": rpc error: code = NotFound desc = could not find container \"bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610\": container with ID starting with bb6974c3f7a26aee546ae462e6e50c539b2ec5e37c627fe6883249bb1d790610 not found: ID does not exist" Jul 01 23:06:44.922044 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:06:44.922003 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="598d38df-446b-4f14-9492-7474419634d6" path="/var/lib/kubelet/pods/598d38df-446b-4f14-9492-7474419634d6/volumes" Jul 01 23:07:06.076575 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.076539 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:07:06.077091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.076831 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="alertmanager" containerID="cri-o://dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9" gracePeriod=120 Jul 01 23:07:06.077091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.076904 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-metric" containerID="cri-o://fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737" gracePeriod=120 Jul 01 23:07:06.077091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.076914 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-web" containerID="cri-o://ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91" gracePeriod=120 Jul 01 23:07:06.077091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.076947 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="prom-label-proxy" containerID="cri-o://6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b" gracePeriod=120 Jul 01 23:07:06.077091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.076967 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="config-reloader" containerID="cri-o://552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1" gracePeriod=120 Jul 01 23:07:06.077091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.077008 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy" containerID="cri-o://9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda" gracePeriod=120 Jul 01 23:07:06.642413 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642381 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b" exitCode=0 Jul 01 23:07:06.642413 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642404 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda" exitCode=0 Jul 01 23:07:06.642413 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642410 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1" exitCode=0 Jul 01 23:07:06.642413 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642415 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9" exitCode=0 Jul 01 23:07:06.642695 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642453 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b"} Jul 01 23:07:06.642695 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda"} Jul 01 23:07:06.642695 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642552 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1"} Jul 01 23:07:06.642695 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:06.642575 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9"} Jul 01 23:07:07.327107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.327083 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.437129 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437046 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-web-config\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437129 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437084 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-out\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437129 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437109 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-cluster-tls-config\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437129 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437130 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-volume\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437149 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437191 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437214 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-web\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437240 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-main-db\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437272 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-428vf\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-kube-api-access-428vf\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.437894 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.437862 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.438913 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-main-tls\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.438958 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-tls-assets\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.438994 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-metrics-client-ca\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.439058 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-trusted-ca-bundle\") pod \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\" (UID: \"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab\") " Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.439300 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-main-db\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.439666 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:07:07.439763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.439726 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:07:07.440114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.439902 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-out" (OuterVolumeSpecName: "config-out") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:07:07.441308 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.440475 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.441308 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.440597 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-volume" (OuterVolumeSpecName: "config-volume") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.441308 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.440683 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-kube-api-access-428vf" (OuterVolumeSpecName: "kube-api-access-428vf") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "kube-api-access-428vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:07:07.441308 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.440759 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.441308 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.440850 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.441669 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.441638 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.442313 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.442290 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:07:07.444336 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.444312 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.450805 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.450781 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-web-config" (OuterVolumeSpecName: "web-config") pod "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" (UID: "b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:07.540025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.539981 2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-volume\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540020 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540032 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540041 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540050 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-428vf\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-kube-api-access-428vf\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540059 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-secret-alertmanager-main-tls\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540067 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-tls-assets\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540076 2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-metrics-client-ca\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540084 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540092 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-web-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540100 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-config-out\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.540241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.540107 2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab-cluster-tls-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:07.650051 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650024 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737" exitCode=0 Jul 01 23:07:07.650051 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650046 2569 generic.go:356] "Generic (PLEG): container finished" podID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerID="ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91" exitCode=0 Jul 01 23:07:07.650218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650104 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.650218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737"} Jul 01 23:07:07.650218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650203 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91"} Jul 01 23:07:07.650307 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650231 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab","Type":"ContainerDied","Data":"15e4be2937475f619694639cfb6461df1669922ebf12ad9c78e115dcc2bead13"} Jul 01 23:07:07.650307 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.650269 2569 scope.go:117] "RemoveContainer" containerID="6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b" Jul 01 23:07:07.658007 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.657990 2569 scope.go:117] "RemoveContainer" containerID="fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737" Jul 01 23:07:07.665129 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.665115 2569 scope.go:117] "RemoveContainer" containerID="9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda" Jul 01 23:07:07.671690 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.671670 2569 scope.go:117] "RemoveContainer" containerID="ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91" Jul 01 23:07:07.676032 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.676011 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:07:07.678815 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.678786 2569 scope.go:117] "RemoveContainer" containerID="552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1" Jul 01 23:07:07.680728 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.680710 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:07:07.685274 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.685259 2569 scope.go:117] "RemoveContainer" containerID="dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9" Jul 01 23:07:07.691893 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.691876 2569 scope.go:117] "RemoveContainer" containerID="56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68" Jul 01 23:07:07.698147 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698131 2569 scope.go:117] "RemoveContainer" containerID="6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b" Jul 01 23:07:07.698374 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.698356 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b\": container with ID starting with 6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b not found: ID does not exist" containerID="6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b" Jul 01 23:07:07.698429 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698383 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b"} err="failed to get container status \"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b\": rpc error: code = NotFound desc = could not find container \"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b\": container with ID starting with 6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b not found: ID does not exist" Jul 01 23:07:07.698429 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698403 2569 scope.go:117] "RemoveContainer" containerID="fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737" Jul 01 23:07:07.698642 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.698627 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737\": container with ID starting with fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737 not found: ID does not exist" containerID="fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737" Jul 01 23:07:07.698710 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698644 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737"} err="failed to get container status \"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737\": rpc error: code = NotFound desc = could not find container \"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737\": container with ID starting with fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737 not found: ID does not exist" Jul 01 23:07:07.698710 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698655 2569 scope.go:117] "RemoveContainer" containerID="9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda" Jul 01 23:07:07.698869 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.698852 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda\": container with ID starting with 9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda not found: ID does not exist" containerID="9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda" Jul 01 23:07:07.698905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698875 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda"} err="failed to get container status \"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda\": rpc error: code = NotFound desc = could not find container \"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda\": container with ID starting with 9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda not found: ID does not exist" Jul 01 23:07:07.698905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.698891 2569 scope.go:117] "RemoveContainer" containerID="ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91" Jul 01 23:07:07.699108 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.699091 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91\": container with ID starting with ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91 not found: ID does not exist" containerID="ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91" Jul 01 23:07:07.699168 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699116 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91"} err="failed to get container status \"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91\": rpc error: code = NotFound desc = could not find container \"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91\": container with ID starting with ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91 not found: ID does not exist" Jul 01 23:07:07.699168 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699137 2569 scope.go:117] "RemoveContainer" containerID="552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1" Jul 01 23:07:07.699375 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.699360 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1\": container with ID starting with 552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1 not found: ID does not exist" containerID="552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1" Jul 01 23:07:07.699413 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699380 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1"} err="failed to get container status \"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1\": rpc error: code = NotFound desc = could not find container \"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1\": container with ID starting with 552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1 not found: ID does not exist" Jul 01 23:07:07.699413 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699393 2569 scope.go:117] "RemoveContainer" containerID="dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9" Jul 01 23:07:07.699635 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.699615 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9\": container with ID starting with dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9 not found: ID does not exist" containerID="dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9" Jul 01 23:07:07.699708 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699643 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9"} err="failed to get container status \"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9\": rpc error: code = NotFound desc = could not find container \"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9\": container with ID starting with dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9 not found: ID does not exist" Jul 01 23:07:07.699708 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699662 2569 scope.go:117] "RemoveContainer" containerID="56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68" Jul 01 23:07:07.699905 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:07.699889 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68\": container with ID starting with 56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68 not found: ID does not exist" containerID="56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68" Jul 01 23:07:07.699945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699909 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68"} err="failed to get container status \"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68\": rpc error: code = NotFound desc = could not find container \"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68\": container with ID starting with 56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68 not found: ID does not exist" Jul 01 23:07:07.699945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.699925 2569 scope.go:117] "RemoveContainer" containerID="6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b" Jul 01 23:07:07.700144 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700127 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b"} err="failed to get container status \"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b\": rpc error: code = NotFound desc = could not find container \"6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b\": container with ID starting with 6b361e2d600f4e453dd348f493228b9727e3b44f4d42fdecc471bc1eeaad502b not found: ID does not exist" Jul 01 23:07:07.700216 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700146 2569 scope.go:117] "RemoveContainer" containerID="fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737" Jul 01 23:07:07.700376 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700357 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737"} err="failed to get container status \"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737\": rpc error: code = NotFound desc = could not find container \"fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737\": container with ID starting with fe6db279aa34d1aae539b5d81e31e82a9bfe3821f19b49025cdf21fb4dd39737 not found: ID does not exist" Jul 01 23:07:07.700422 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700377 2569 scope.go:117] "RemoveContainer" containerID="9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda" Jul 01 23:07:07.700552 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700534 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda"} err="failed to get container status \"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda\": rpc error: code = NotFound desc = could not find container \"9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda\": container with ID starting with 9c11a81623f80483eea6a66e7a70d5062b0a3d9f11617f38c15374351c403dda not found: ID does not exist" Jul 01 23:07:07.700618 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700553 2569 scope.go:117] "RemoveContainer" containerID="ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91" Jul 01 23:07:07.700823 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700800 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91"} err="failed to get container status \"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91\": rpc error: code = NotFound desc = could not find container \"ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91\": container with ID starting with ec9a71fdc3a6aaecb06ddfc2fdc1356bf2983d2945c2a50243f9ec781629cf91 not found: ID does not exist" Jul 01 23:07:07.700823 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.700827 2569 scope.go:117] "RemoveContainer" containerID="552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1" Jul 01 23:07:07.701025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.701005 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1"} err="failed to get container status \"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1\": rpc error: code = NotFound desc = could not find container \"552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1\": container with ID starting with 552696fc97820fb01d48ba2ca3d6df1b9419c0a2a79b6d7eb7d2696666dc91b1 not found: ID does not exist" Jul 01 23:07:07.701025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.701026 2569 scope.go:117] "RemoveContainer" containerID="dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9" Jul 01 23:07:07.701252 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.701228 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9"} err="failed to get container status \"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9\": rpc error: code = NotFound desc = could not find container \"dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9\": container with ID starting with dfa604cb3d62d748c28a56c4194bca96d6f63916353226673ed52c45545c75b9 not found: ID does not exist" Jul 01 23:07:07.701252 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.701252 2569 scope.go:117] "RemoveContainer" containerID="56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68" Jul 01 23:07:07.701505 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.701482 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68"} err="failed to get container status \"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68\": rpc error: code = NotFound desc = could not find container \"56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68\": container with ID starting with 56214f4c5d1fe26efde86635397194d66d26a2ce6e30325098fe7530ec366f68 not found: ID does not exist" Jul 01 23:07:07.710697 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710673 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:07:07.710964 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710952 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="prom-label-proxy" Jul 01 23:07:07.711002 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710967 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="prom-label-proxy" Jul 01 23:07:07.711002 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710980 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="init-config-reloader" Jul 01 23:07:07.711002 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710986 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="init-config-reloader" Jul 01 23:07:07.711002 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710993 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy" Jul 01 23:07:07.711002 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.710998 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711011 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="config-reloader" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711017 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="config-reloader" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711023 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-metric" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711029 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-metric" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711037 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598d38df-446b-4f14-9492-7474419634d6" containerName="console" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711042 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="598d38df-446b-4f14-9492-7474419634d6" containerName="console" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711051 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="alertmanager" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711056 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="alertmanager" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711061 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-web" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711066 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-web" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711112 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="598d38df-446b-4f14-9492-7474419634d6" containerName="console" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711119 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="alertmanager" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711124 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="config-reloader" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711130 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711137 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-metric" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711143 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="prom-label-proxy" Jul 01 23:07:07.711145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.711148 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" containerName="kube-rbac-proxy-web" Jul 01 23:07:07.714753 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.714739 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.717149 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.717129 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 01 23:07:07.717226 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.717165 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 01 23:07:07.717226 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.717207 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 01 23:07:07.719919 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.719901 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 01 23:07:07.719994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.719905 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 01 23:07:07.719994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.719947 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 01 23:07:07.720163 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.720146 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jg4qp\"" Jul 01 23:07:07.720242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.720229 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 01 23:07:07.720484 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.720461 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 01 23:07:07.725362 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.725345 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 01 23:07:07.735816 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.735796 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:07:07.842042 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842015 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842042 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842043 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842196 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842065 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842196 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842080 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842196 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842135 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-web-config\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842196 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdc4d633-9b66-49db-beb6-282be6f36a35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842205 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdc4d633-9b66-49db-beb6-282be6f36a35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842222 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdc4d633-9b66-49db-beb6-282be6f36a35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842241 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cdc4d633-9b66-49db-beb6-282be6f36a35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842268 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-config-volume\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842282 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffkfl\" (UniqueName: \"kubernetes.io/projected/cdc4d633-9b66-49db-beb6-282be6f36a35-kube-api-access-ffkfl\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842325 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842311 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.842525 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.842336 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdc4d633-9b66-49db-beb6-282be6f36a35-config-out\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943137 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943070 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943137 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943137 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943121 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943313 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943313 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943275 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-web-config\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943313 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdc4d633-9b66-49db-beb6-282be6f36a35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943452 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdc4d633-9b66-49db-beb6-282be6f36a35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943452 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943367 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdc4d633-9b66-49db-beb6-282be6f36a35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943452 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943391 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cdc4d633-9b66-49db-beb6-282be6f36a35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943452 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943431 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-config-volume\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943457 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffkfl\" (UniqueName: \"kubernetes.io/projected/cdc4d633-9b66-49db-beb6-282be6f36a35-kube-api-access-ffkfl\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943496 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.943661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.943527 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdc4d633-9b66-49db-beb6-282be6f36a35-config-out\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.944667 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.944340 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdc4d633-9b66-49db-beb6-282be6f36a35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.944667 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.944594 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdc4d633-9b66-49db-beb6-282be6f36a35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.945814 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.945789 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cdc4d633-9b66-49db-beb6-282be6f36a35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.946207 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.946181 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.946316 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.946264 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.946433 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.946410 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdc4d633-9b66-49db-beb6-282be6f36a35-config-out\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.946511 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.946420 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.946511 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.946493 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.946937 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.946911 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-web-config\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.947032 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.947016 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-config-volume\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.947722 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.947703 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdc4d633-9b66-49db-beb6-282be6f36a35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.947916 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.947900 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cdc4d633-9b66-49db-beb6-282be6f36a35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:07.954888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:07.954867 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffkfl\" (UniqueName: \"kubernetes.io/projected/cdc4d633-9b66-49db-beb6-282be6f36a35-kube-api-access-ffkfl\") pod \"alertmanager-main-0\" (UID: \"cdc4d633-9b66-49db-beb6-282be6f36a35\") " pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:08.024017 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:08.023994 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 01 23:07:08.160963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:08.160935 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 01 23:07:08.164318 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:07:08.164293 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc4d633_9b66_49db_beb6_282be6f36a35.slice/crio-65daa45553456872439b876cbe63996d55ed0e5ecf34ee4b50b89ca4adc94255 WatchSource:0}: Error finding container 65daa45553456872439b876cbe63996d55ed0e5ecf34ee4b50b89ca4adc94255: Status 404 returned error can't find the container with id 65daa45553456872439b876cbe63996d55ed0e5ecf34ee4b50b89ca4adc94255 Jul 01 23:07:08.656537 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:08.656501 2569 generic.go:356] "Generic (PLEG): container finished" podID="cdc4d633-9b66-49db-beb6-282be6f36a35" containerID="c80f69bfd5247b95a4bc2547e679b6efda9461e7c5b0909fe00a8ba46933be81" exitCode=0 Jul 01 23:07:08.656989 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:08.656550 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerDied","Data":"c80f69bfd5247b95a4bc2547e679b6efda9461e7c5b0909fe00a8ba46933be81"} Jul 01 23:07:08.656989 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:08.656631 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"65daa45553456872439b876cbe63996d55ed0e5ecf34ee4b50b89ca4adc94255"} Jul 01 23:07:08.923955 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:08.923929 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab" path="/var/lib/kubelet/pods/b5efcfbd-532f-4e20-a5dd-4a09ec5f3eab/volumes" Jul 01 23:07:09.664505 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.664472 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"484705c3324b3ef7b3f971265446debc69b424a35088517011140672dc18baa3"} Jul 01 23:07:09.664885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.664534 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"ccfcb28cc37dfc7c28d396d4bf59a0348e37edf8f183ce9598430c9426179773"} Jul 01 23:07:09.664885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.664563 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"b71f2fbcb7f77e1ef9d00860155f8c02e580ad4aaea08b3021dbf60f5dcd026c"} Jul 01 23:07:09.664885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.664588 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"02e084bef1e10202294232f6b2e9017139f76d162a98dbbd80700926bf744a70"} Jul 01 23:07:09.664885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.664632 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"d7e3b3551067f32fede1f5fcd39015f0a85e70bdc6893d2a1bfb291ab8ed472e"} Jul 01 23:07:09.664885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.664656 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cdc4d633-9b66-49db-beb6-282be6f36a35","Type":"ContainerStarted","Data":"710a0167515e8d977324be18a033d31657312e2847c7bfe7e9e362565cdac6d1"} Jul 01 23:07:09.695113 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:09.695067 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.680353395 podStartE2EDuration="2.695052764s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.699369ms" isStatefulPod=false podCreationTimestamp="2026-07-01 23:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:07:09.693480992 +0000 UTC m=+159.916009221" watchObservedRunningTime="2026-07-01 23:07:09.695052764 +0000 UTC m=+159.917580973" Jul 01 23:07:10.133797 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.133759 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk"] Jul 01 23:07:10.136264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.136243 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.139262 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.139233 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 01 23:07:10.139387 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.139337 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z2j64\"" Jul 01 23:07:10.139451 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.139401 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 01 23:07:10.139451 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.139435 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 01 23:07:10.139555 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.139473 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 01 23:07:10.139555 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.139554 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 01 23:07:10.145116 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.145099 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 01 23:07:10.152759 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.152738 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk"] Jul 01 23:07:10.262728 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-telemeter-client-tls\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.262905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262736 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-serving-certs-ca-bundle\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.262905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262762 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-secret-telemeter-client\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.262905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262853 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-metrics-client-ca\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.262905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-federate-client-tls\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.263042 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262913 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.263042 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262956 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7llzt\" (UniqueName: \"kubernetes.io/projected/b4287f19-1583-44c5-aa41-380e32238c0e-kube-api-access-7llzt\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.263042 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.262990 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364202 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-metrics-client-ca\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-federate-client-tls\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364289 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364332 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7llzt\" (UniqueName: \"kubernetes.io/projected/b4287f19-1583-44c5-aa41-380e32238c0e-kube-api-access-7llzt\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-telemeter-client-tls\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-serving-certs-ca-bundle\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.364477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.364468 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-secret-telemeter-client\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.365176 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.365145 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-metrics-client-ca\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.365285 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.365217 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.365285 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.365266 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4287f19-1583-44c5-aa41-380e32238c0e-serving-certs-ca-bundle\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.367477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.367452 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-federate-client-tls\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.367558 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.367452 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-telemeter-client-tls\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.367558 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.367502 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.367639 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.367558 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b4287f19-1583-44c5-aa41-380e32238c0e-secret-telemeter-client\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.372929 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.372907 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7llzt\" (UniqueName: \"kubernetes.io/projected/b4287f19-1583-44c5-aa41-380e32238c0e-kube-api-access-7llzt\") pod \"telemeter-client-74bcbbdbf8-pv8hk\" (UID: \"b4287f19-1583-44c5-aa41-380e32238c0e\") " pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.447381 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.447286 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" Jul 01 23:07:10.591486 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.590847 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk"] Jul 01 23:07:10.593069 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:07:10.593036 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4287f19_1583_44c5_aa41_380e32238c0e.slice/crio-6dd0255b6d57ac4130c9286ec725048a118594f76f71d5d65273ecd02a973f73 WatchSource:0}: Error finding container 6dd0255b6d57ac4130c9286ec725048a118594f76f71d5d65273ecd02a973f73: Status 404 returned error can't find the container with id 6dd0255b6d57ac4130c9286ec725048a118594f76f71d5d65273ecd02a973f73 Jul 01 23:07:10.670908 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:10.670869 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" event={"ID":"b4287f19-1583-44c5-aa41-380e32238c0e","Type":"ContainerStarted","Data":"6dd0255b6d57ac4130c9286ec725048a118594f76f71d5d65273ecd02a973f73"} Jul 01 23:07:12.680515 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:12.680486 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" event={"ID":"b4287f19-1583-44c5-aa41-380e32238c0e","Type":"ContainerStarted","Data":"b4f373ace55f4f28e29bb60401e24543c0e0edd001403514a411b5fcd3c5ab35"} Jul 01 23:07:13.686906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:13.686870 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" event={"ID":"b4287f19-1583-44c5-aa41-380e32238c0e","Type":"ContainerStarted","Data":"39b7a1981aa3041d87e69cf5a726028fcc1272f82aee0d768ad79da14f8616cf"} Jul 01 23:07:13.687320 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:13.686929 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" event={"ID":"b4287f19-1583-44c5-aa41-380e32238c0e","Type":"ContainerStarted","Data":"6be706ed811d22505fce231a1ce77065c8d65b50d2875447c3be60f497832c5c"} Jul 01 23:07:13.711823 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:13.711773 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74bcbbdbf8-pv8hk" podStartSLOduration=1.683826676 podStartE2EDuration="3.711757911s" totalImagesPullingTime="2.027931235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:07:13.710470368 +0000 UTC m=+163.932998581" watchObservedRunningTime="2026-07-01 23:07:13.711757911 +0000 UTC m=+163.934286121" Jul 01 23:07:14.516844 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.516807 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7598fcc659-p6brv"] Jul 01 23:07:14.519216 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.519192 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.535352 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.535327 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7598fcc659-p6brv"] Jul 01 23:07:14.597540 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597510 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-oauth-config\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.597739 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597546 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-oauth-serving-cert\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.597739 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597565 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-trusted-ca-bundle\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.597739 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597593 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-serving-cert\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.597739 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597686 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-service-ca\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.597916 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597749 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-config\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.597916 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.597774 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncz8s\" (UniqueName: \"kubernetes.io/projected/0e3babf4-8f94-4a45-b97f-9902fa15a312-kube-api-access-ncz8s\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.698885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698851 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-oauth-config\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.698885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698887 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-oauth-serving-cert\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699377 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-trusted-ca-bundle\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699377 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698921 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-serving-cert\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699377 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698944 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-service-ca\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699377 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698980 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-config\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699377 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.698996 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncz8s\" (UniqueName: \"kubernetes.io/projected/0e3babf4-8f94-4a45-b97f-9902fa15a312-kube-api-access-ncz8s\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699854 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.699792 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-oauth-serving-cert\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.699865 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-service-ca\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.699884 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-trusted-ca-bundle\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.699970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.699895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-config\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.701329 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.701305 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-oauth-config\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.701439 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.701422 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-serving-cert\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.710483 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.710461 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncz8s\" (UniqueName: \"kubernetes.io/projected/0e3babf4-8f94-4a45-b97f-9902fa15a312-kube-api-access-ncz8s\") pod \"console-7598fcc659-p6brv\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.829668 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.829634 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:14.960564 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:14.960533 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7598fcc659-p6brv"] Jul 01 23:07:14.963490 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:07:14.963469 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3babf4_8f94_4a45_b97f_9902fa15a312.slice/crio-e17099f8dc88c5c1e4c542383efdcb3c4b47f2f632c6cbf5b59c786afc1176c8 WatchSource:0}: Error finding container e17099f8dc88c5c1e4c542383efdcb3c4b47f2f632c6cbf5b59c786afc1176c8: Status 404 returned error can't find the container with id e17099f8dc88c5c1e4c542383efdcb3c4b47f2f632c6cbf5b59c786afc1176c8 Jul 01 23:07:15.696757 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:15.696725 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7598fcc659-p6brv" event={"ID":"0e3babf4-8f94-4a45-b97f-9902fa15a312","Type":"ContainerStarted","Data":"e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159"} Jul 01 23:07:15.696937 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:15.696770 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7598fcc659-p6brv" event={"ID":"0e3babf4-8f94-4a45-b97f-9902fa15a312","Type":"ContainerStarted","Data":"e17099f8dc88c5c1e4c542383efdcb3c4b47f2f632c6cbf5b59c786afc1176c8"} Jul 01 23:07:15.716839 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:15.716795 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7598fcc659-p6brv" podStartSLOduration=1.716779377 podStartE2EDuration="1.716779377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:07:15.715786018 +0000 UTC m=+165.938314272" watchObservedRunningTime="2026-07-01 23:07:15.716779377 +0000 UTC m=+165.939307587" Jul 01 23:07:24.830066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:24.829975 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:24.830066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:24.830051 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:24.834334 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:24.834311 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:25.739286 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:25.739257 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:07:25.803562 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:25.803532 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8696cd744d-jn8zj"] Jul 01 23:07:50.829776 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:50.829707 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8696cd744d-jn8zj" podUID="8c891f31-5850-4014-b72c-e6a34befa288" containerName="console" containerID="cri-o://a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676" gracePeriod=15 Jul 01 23:07:51.067333 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.067310 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:07:51.087062 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.086969 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdslx\" (UniqueName: \"kubernetes.io/projected/8c891f31-5850-4014-b72c-e6a34befa288-kube-api-access-wdslx\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087062 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087026 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-trusted-ca-bundle\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087076 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-oauth-config\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087122 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-oauth-serving-cert\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087160 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-console-config\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087184 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-serving-cert\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087228 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-service-ca\") pod \"8c891f31-5850-4014-b72c-e6a34befa288\" (UID: \"8c891f31-5850-4014-b72c-e6a34befa288\") " Jul 01 23:07:51.087962 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087741 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-console-config" (OuterVolumeSpecName: "console-config") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:07:51.087962 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.087792 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:07:51.088105 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.088002 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:07:51.088105 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.088008 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:07:51.089815 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.089788 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:51.089815 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.089797 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c891f31-5850-4014-b72c-e6a34befa288-kube-api-access-wdslx" (OuterVolumeSpecName: "kube-api-access-wdslx") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "kube-api-access-wdslx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:07:51.090112 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.090072 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c891f31-5850-4014-b72c-e6a34befa288" (UID: "8c891f31-5850-4014-b72c-e6a34befa288"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:07:51.188693 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188663 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdslx\" (UniqueName: \"kubernetes.io/projected/8c891f31-5850-4014-b72c-e6a34befa288-kube-api-access-wdslx\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.188693 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188687 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-trusted-ca-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.188693 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188697 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-oauth-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.188907 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188705 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-oauth-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.188907 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188713 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-console-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.188907 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188721 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c891f31-5850-4014-b72c-e6a34befa288-console-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.188907 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.188729 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c891f31-5850-4014-b72c-e6a34befa288-service-ca\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:07:51.845430 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.845394 2569 generic.go:356] "Generic (PLEG): container finished" podID="8c891f31-5850-4014-b72c-e6a34befa288" containerID="a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676" exitCode=0 Jul 01 23:07:51.845858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.845451 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8696cd744d-jn8zj" Jul 01 23:07:51.845858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.845475 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696cd744d-jn8zj" event={"ID":"8c891f31-5850-4014-b72c-e6a34befa288","Type":"ContainerDied","Data":"a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676"} Jul 01 23:07:51.845858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.845546 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8696cd744d-jn8zj" event={"ID":"8c891f31-5850-4014-b72c-e6a34befa288","Type":"ContainerDied","Data":"2ee1e82496788c12004acb7c45306ddd4c2031543f0577132f9d577c73c7b643"} Jul 01 23:07:51.845858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.845567 2569 scope.go:117] "RemoveContainer" containerID="a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676" Jul 01 23:07:51.856047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.856029 2569 scope.go:117] "RemoveContainer" containerID="a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676" Jul 01 23:07:51.856327 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:07:51.856300 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676\": container with ID starting with a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676 not found: ID does not exist" containerID="a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676" Jul 01 23:07:51.856407 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.856335 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676"} err="failed to get container status \"a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676\": rpc error: code = NotFound desc = could not find container \"a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676\": container with ID starting with a09b2cafde78cc376f8c8af69d939f8c3000fe99812b43724e460ca472eed676 not found: ID does not exist" Jul 01 23:07:51.869933 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.869913 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8696cd744d-jn8zj"] Jul 01 23:07:51.874131 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:51.874113 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8696cd744d-jn8zj"] Jul 01 23:07:52.920961 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:07:52.920930 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c891f31-5850-4014-b72c-e6a34befa288" path="/var/lib/kubelet/pods/8c891f31-5850-4014-b72c-e6a34befa288/volumes" Jul 01 23:08:24.869342 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.869311 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d65b8d58d-slbgt"] Jul 01 23:08:24.869948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.869849 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c891f31-5850-4014-b72c-e6a34befa288" containerName="console" Jul 01 23:08:24.869948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.869869 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c891f31-5850-4014-b72c-e6a34befa288" containerName="console" Jul 01 23:08:24.870070 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.869963 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c891f31-5850-4014-b72c-e6a34befa288" containerName="console" Jul 01 23:08:24.872899 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.872873 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.884387 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.884368 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d65b8d58d-slbgt"] Jul 01 23:08:24.960080 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960046 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-oauth-config\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.960241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960086 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdsfk\" (UniqueName: \"kubernetes.io/projected/95789aac-e27e-47a6-818f-562c2d53e765-kube-api-access-tdsfk\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.960241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960125 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-service-ca\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.960241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960147 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-oauth-serving-cert\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.960241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960176 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-trusted-ca-bundle\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.960241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960222 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-console-config\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:24.960422 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:24.960275 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-serving-cert\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.061624 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.061576 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-service-ca\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.061813 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.061756 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-oauth-serving-cert\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.061967 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.061943 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-trusted-ca-bundle\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062041 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062014 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-console-config\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062182 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062160 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-serving-cert\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062276 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062242 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-oauth-config\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062336 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-service-ca\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdsfk\" (UniqueName: \"kubernetes.io/projected/95789aac-e27e-47a6-818f-562c2d53e765-kube-api-access-tdsfk\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062485 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062406 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-oauth-serving-cert\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062684 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062648 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-console-config\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.062819 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.062798 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-trusted-ca-bundle\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.064708 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.064686 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-oauth-config\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.064830 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.064808 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-serving-cert\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.070336 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.070317 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdsfk\" (UniqueName: \"kubernetes.io/projected/95789aac-e27e-47a6-818f-562c2d53e765-kube-api-access-tdsfk\") pod \"console-5d65b8d58d-slbgt\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.181740 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.181672 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:25.314364 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.314338 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d65b8d58d-slbgt"] Jul 01 23:08:25.317219 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:08:25.317193 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95789aac_e27e_47a6_818f_562c2d53e765.slice/crio-e2500b3240f87ff920a66671b1608946b657055b08277ee4b0e3624c37b7860a WatchSource:0}: Error finding container e2500b3240f87ff920a66671b1608946b657055b08277ee4b0e3624c37b7860a: Status 404 returned error can't find the container with id e2500b3240f87ff920a66671b1608946b657055b08277ee4b0e3624c37b7860a Jul 01 23:08:25.975588 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.975548 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d65b8d58d-slbgt" event={"ID":"95789aac-e27e-47a6-818f-562c2d53e765","Type":"ContainerStarted","Data":"b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d"} Jul 01 23:08:25.975981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.975623 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d65b8d58d-slbgt" event={"ID":"95789aac-e27e-47a6-818f-562c2d53e765","Type":"ContainerStarted","Data":"e2500b3240f87ff920a66671b1608946b657055b08277ee4b0e3624c37b7860a"} Jul 01 23:08:25.994963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:25.994913 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d65b8d58d-slbgt" podStartSLOduration=1.994897734 podStartE2EDuration="1.994897734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:08:25.993474579 +0000 UTC m=+236.216002805" watchObservedRunningTime="2026-07-01 23:08:25.994897734 +0000 UTC m=+236.217425944" Jul 01 23:08:35.181792 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:35.181759 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:35.182198 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:35.181815 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:35.186257 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:35.186237 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:36.014650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:36.014624 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:08:36.061415 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:08:36.061387 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7598fcc659-p6brv"] Jul 01 23:09:01.082292 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.082207 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7598fcc659-p6brv" podUID="0e3babf4-8f94-4a45-b97f-9902fa15a312" containerName="console" containerID="cri-o://e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159" gracePeriod=15 Jul 01 23:09:01.326543 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.326518 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:09:01.461983 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.461893 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-config\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.461983 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.461949 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-service-ca\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.461983 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.461976 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-oauth-serving-cert\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.462214 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462021 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-serving-cert\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.462214 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462050 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-trusted-ca-bundle\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.462214 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462113 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-oauth-config\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.462214 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462148 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncz8s\" (UniqueName: \"kubernetes.io/projected/0e3babf4-8f94-4a45-b97f-9902fa15a312-kube-api-access-ncz8s\") pod \"0e3babf4-8f94-4a45-b97f-9902fa15a312\" (UID: \"0e3babf4-8f94-4a45-b97f-9902fa15a312\") " Jul 01 23:09:01.462412 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462363 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-config" (OuterVolumeSpecName: "console-config") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:09:01.462470 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462445 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:09:01.462524 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462475 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:09:01.462578 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.462503 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:09:01.464225 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.464199 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:09:01.464309 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.464223 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:09:01.464342 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.464316 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3babf4-8f94-4a45-b97f-9902fa15a312-kube-api-access-ncz8s" (OuterVolumeSpecName: "kube-api-access-ncz8s") pod "0e3babf4-8f94-4a45-b97f-9902fa15a312" (UID: "0e3babf4-8f94-4a45-b97f-9902fa15a312"). InnerVolumeSpecName "kube-api-access-ncz8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:09:01.563349 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563320 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-oauth-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:01.563349 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563341 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncz8s\" (UniqueName: \"kubernetes.io/projected/0e3babf4-8f94-4a45-b97f-9902fa15a312-kube-api-access-ncz8s\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:01.563349 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563352 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:01.563552 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563361 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-service-ca\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:01.563552 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563368 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-oauth-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:01.563552 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563377 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e3babf4-8f94-4a45-b97f-9902fa15a312-console-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:01.563552 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:01.563385 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e3babf4-8f94-4a45-b97f-9902fa15a312-trusted-ca-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:02.110813 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.110782 2569 generic.go:356] "Generic (PLEG): container finished" podID="0e3babf4-8f94-4a45-b97f-9902fa15a312" containerID="e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159" exitCode=0 Jul 01 23:09:02.111269 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.110833 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7598fcc659-p6brv" Jul 01 23:09:02.111269 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.110859 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7598fcc659-p6brv" event={"ID":"0e3babf4-8f94-4a45-b97f-9902fa15a312","Type":"ContainerDied","Data":"e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159"} Jul 01 23:09:02.111269 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.110913 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7598fcc659-p6brv" event={"ID":"0e3babf4-8f94-4a45-b97f-9902fa15a312","Type":"ContainerDied","Data":"e17099f8dc88c5c1e4c542383efdcb3c4b47f2f632c6cbf5b59c786afc1176c8"} Jul 01 23:09:02.111269 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.110933 2569 scope.go:117] "RemoveContainer" containerID="e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159" Jul 01 23:09:02.121315 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.121299 2569 scope.go:117] "RemoveContainer" containerID="e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159" Jul 01 23:09:02.121658 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:09:02.121637 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159\": container with ID starting with e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159 not found: ID does not exist" containerID="e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159" Jul 01 23:09:02.121741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.121670 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159"} err="failed to get container status \"e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159\": rpc error: code = NotFound desc = could not find container \"e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159\": container with ID starting with e4df1f4e903ec27be5aa8228904b73bcc37a654dbbf9821146970325ba9c2159 not found: ID does not exist" Jul 01 23:09:02.139520 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.139492 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7598fcc659-p6brv"] Jul 01 23:09:02.145451 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.145428 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7598fcc659-p6brv"] Jul 01 23:09:02.158253 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.158232 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z"] Jul 01 23:09:02.158546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.158535 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3babf4-8f94-4a45-b97f-9902fa15a312" containerName="console" Jul 01 23:09:02.158590 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.158549 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3babf4-8f94-4a45-b97f-9902fa15a312" containerName="console" Jul 01 23:09:02.158643 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.158620 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3babf4-8f94-4a45-b97f-9902fa15a312" containerName="console" Jul 01 23:09:02.163008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.162992 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.166426 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.166014 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 01 23:09:02.166426 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.166332 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6j5mm\"" Jul 01 23:09:02.166426 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.166391 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 01 23:09:02.177342 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.177316 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z"] Jul 01 23:09:02.269561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.269527 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.269757 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.269574 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.269757 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.269600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bprs\" (UniqueName: \"kubernetes.io/projected/e86cc8c8-d781-4688-a4b9-ffc24f07306e-kube-api-access-2bprs\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.370936 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.370850 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.370936 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.370896 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.370936 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.370931 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bprs\" (UniqueName: \"kubernetes.io/projected/e86cc8c8-d781-4688-a4b9-ffc24f07306e-kube-api-access-2bprs\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.371241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.371221 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.371326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.371306 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.380124 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.380104 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bprs\" (UniqueName: \"kubernetes.io/projected/e86cc8c8-d781-4688-a4b9-ffc24f07306e-kube-api-access-2bprs\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.476123 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.476090 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:02.606746 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.606699 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z"] Jul 01 23:09:02.609370 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:02.609342 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86cc8c8_d781_4688_a4b9_ffc24f07306e.slice/crio-798680a88e8537ee4b44d90301bbdc14660cb90fa70907245d335fd0377216f6 WatchSource:0}: Error finding container 798680a88e8537ee4b44d90301bbdc14660cb90fa70907245d335fd0377216f6: Status 404 returned error can't find the container with id 798680a88e8537ee4b44d90301bbdc14660cb90fa70907245d335fd0377216f6 Jul 01 23:09:02.921625 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:02.921565 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e3babf4-8f94-4a45-b97f-9902fa15a312" path="/var/lib/kubelet/pods/0e3babf4-8f94-4a45-b97f-9902fa15a312/volumes" Jul 01 23:09:03.118226 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:03.118192 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" event={"ID":"e86cc8c8-d781-4688-a4b9-ffc24f07306e","Type":"ContainerStarted","Data":"798680a88e8537ee4b44d90301bbdc14660cb90fa70907245d335fd0377216f6"} Jul 01 23:09:11.152538 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:11.152501 2569 generic.go:356] "Generic (PLEG): container finished" podID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerID="14f3de30e9d6b526607628bf6a40b11a64516328c13a3b3fa61fc89c2e6ec96d" exitCode=0 Jul 01 23:09:11.152940 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:11.152589 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" event={"ID":"e86cc8c8-d781-4688-a4b9-ffc24f07306e","Type":"ContainerDied","Data":"14f3de30e9d6b526607628bf6a40b11a64516328c13a3b3fa61fc89c2e6ec96d"} Jul 01 23:09:14.166515 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:14.166480 2569 generic.go:356] "Generic (PLEG): container finished" podID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerID="042ff222b0a75d3481ca6750153756bd381251a164513008a3f3b54a2b94576d" exitCode=0 Jul 01 23:09:14.166908 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:14.166559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" event={"ID":"e86cc8c8-d781-4688-a4b9-ffc24f07306e","Type":"ContainerDied","Data":"042ff222b0a75d3481ca6750153756bd381251a164513008a3f3b54a2b94576d"} Jul 01 23:09:21.199226 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:21.199133 2569 generic.go:356] "Generic (PLEG): container finished" podID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerID="a9aaff6d4ba4061c7c1dbd64e8acb318dd152ad169a15d60b3ee41460b3c2ff6" exitCode=0 Jul 01 23:09:21.199571 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:21.199217 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" event={"ID":"e86cc8c8-d781-4688-a4b9-ffc24f07306e","Type":"ContainerDied","Data":"a9aaff6d4ba4061c7c1dbd64e8acb318dd152ad169a15d60b3ee41460b3c2ff6"} Jul 01 23:09:22.325904 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.325873 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:22.444225 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.444186 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-bundle\") pod \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " Jul 01 23:09:22.444225 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.444229 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-util\") pod \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " Jul 01 23:09:22.444466 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.444310 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bprs\" (UniqueName: \"kubernetes.io/projected/e86cc8c8-d781-4688-a4b9-ffc24f07306e-kube-api-access-2bprs\") pod \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\" (UID: \"e86cc8c8-d781-4688-a4b9-ffc24f07306e\") " Jul 01 23:09:22.444872 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.444846 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-bundle" (OuterVolumeSpecName: "bundle") pod "e86cc8c8-d781-4688-a4b9-ffc24f07306e" (UID: "e86cc8c8-d781-4688-a4b9-ffc24f07306e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:09:22.446460 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.446433 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86cc8c8-d781-4688-a4b9-ffc24f07306e-kube-api-access-2bprs" (OuterVolumeSpecName: "kube-api-access-2bprs") pod "e86cc8c8-d781-4688-a4b9-ffc24f07306e" (UID: "e86cc8c8-d781-4688-a4b9-ffc24f07306e"). InnerVolumeSpecName "kube-api-access-2bprs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:09:22.448347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.448323 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-util" (OuterVolumeSpecName: "util") pod "e86cc8c8-d781-4688-a4b9-ffc24f07306e" (UID: "e86cc8c8-d781-4688-a4b9-ffc24f07306e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:09:22.545202 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.545099 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bprs\" (UniqueName: \"kubernetes.io/projected/e86cc8c8-d781-4688-a4b9-ffc24f07306e-kube-api-access-2bprs\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:22.545202 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.545143 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:22.545202 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:22.545153 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e86cc8c8-d781-4688-a4b9-ffc24f07306e-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:23.208518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:23.208440 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" event={"ID":"e86cc8c8-d781-4688-a4b9-ffc24f07306e","Type":"ContainerDied","Data":"798680a88e8537ee4b44d90301bbdc14660cb90fa70907245d335fd0377216f6"} Jul 01 23:09:23.208518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:23.208470 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d9ch5z" Jul 01 23:09:23.208518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:23.208490 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="798680a88e8537ee4b44d90301bbdc14660cb90fa70907245d335fd0377216f6" Jul 01 23:09:29.935185 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935146 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx"] Jul 01 23:09:29.935769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935626 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="util" Jul 01 23:09:29.935769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935678 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="util" Jul 01 23:09:29.935769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935701 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="pull" Jul 01 23:09:29.935769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935710 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="pull" Jul 01 23:09:29.935769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935722 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="extract" Jul 01 23:09:29.935769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935731 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="extract" Jul 01 23:09:29.936073 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.935858 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86cc8c8-d781-4688-a4b9-ffc24f07306e" containerName="extract" Jul 01 23:09:29.939845 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.939823 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:29.942144 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.942120 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 01 23:09:29.942292 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.942221 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 01 23:09:29.942292 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.942238 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-6z6q5\"" Jul 01 23:09:29.952259 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:29.952237 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx"] Jul 01 23:09:30.106022 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.105996 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c99883b-a25d-49ab-9e44-8d5e1ddb2167-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-d45bx\" (UID: \"2c99883b-a25d-49ab-9e44-8d5e1ddb2167\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.106176 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.106053 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5wz6\" (UniqueName: \"kubernetes.io/projected/2c99883b-a25d-49ab-9e44-8d5e1ddb2167-kube-api-access-m5wz6\") pod \"cert-manager-operator-controller-manager-57757f7cfc-d45bx\" (UID: \"2c99883b-a25d-49ab-9e44-8d5e1ddb2167\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.206716 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.206646 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5wz6\" (UniqueName: \"kubernetes.io/projected/2c99883b-a25d-49ab-9e44-8d5e1ddb2167-kube-api-access-m5wz6\") pod \"cert-manager-operator-controller-manager-57757f7cfc-d45bx\" (UID: \"2c99883b-a25d-49ab-9e44-8d5e1ddb2167\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.206716 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.206704 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c99883b-a25d-49ab-9e44-8d5e1ddb2167-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-d45bx\" (UID: \"2c99883b-a25d-49ab-9e44-8d5e1ddb2167\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.206998 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.206983 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c99883b-a25d-49ab-9e44-8d5e1ddb2167-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-d45bx\" (UID: \"2c99883b-a25d-49ab-9e44-8d5e1ddb2167\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.215573 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.215555 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 01 23:09:30.225836 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.225815 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 01 23:09:30.236961 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.236935 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5wz6\" (UniqueName: \"kubernetes.io/projected/2c99883b-a25d-49ab-9e44-8d5e1ddb2167-kube-api-access-m5wz6\") pod \"cert-manager-operator-controller-manager-57757f7cfc-d45bx\" (UID: \"2c99883b-a25d-49ab-9e44-8d5e1ddb2167\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.247224 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.247202 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:09:30.247332 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.247304 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:09:30.251262 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.251242 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-6z6q5\"" Jul 01 23:09:30.253555 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.253539 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 23:09:30.259261 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.259246 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" Jul 01 23:09:30.418155 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:30.418093 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx"] Jul 01 23:09:30.421873 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:30.421840 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c99883b_a25d_49ab_9e44_8d5e1ddb2167.slice/crio-38c671fadb42e56efce879abb1d197691b9673eb82fa1821a55214c64f6b762a WatchSource:0}: Error finding container 38c671fadb42e56efce879abb1d197691b9673eb82fa1821a55214c64f6b762a: Status 404 returned error can't find the container with id 38c671fadb42e56efce879abb1d197691b9673eb82fa1821a55214c64f6b762a Jul 01 23:09:31.244258 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:31.244185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" event={"ID":"2c99883b-a25d-49ab-9e44-8d5e1ddb2167","Type":"ContainerStarted","Data":"38c671fadb42e56efce879abb1d197691b9673eb82fa1821a55214c64f6b762a"} Jul 01 23:09:32.250432 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:32.250352 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" event={"ID":"2c99883b-a25d-49ab-9e44-8d5e1ddb2167","Type":"ContainerStarted","Data":"82493e51ef6190989c0e672247955e3e5f1b3675903a9bb54cebc5f07b16a75b"} Jul 01 23:09:32.275054 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:32.275014 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-d45bx" podStartSLOduration=1.8328605 podStartE2EDuration="3.274997794s" totalImagesPullingTime="1.442137294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:09:32.274023269 +0000 UTC m=+302.496551478" watchObservedRunningTime="2026-07-01 23:09:32.274997794 +0000 UTC m=+302.497526003" Jul 01 23:09:33.592538 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.592508 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd"] Jul 01 23:09:33.595388 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.595369 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.597519 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.597501 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 01 23:09:33.597655 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.597636 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6j5mm\"" Jul 01 23:09:33.598343 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.598321 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 01 23:09:33.606431 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.606408 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd"] Jul 01 23:09:33.635452 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.635429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p29t\" (UniqueName: \"kubernetes.io/projected/3c803c03-adfc-41bd-bc99-920f57cc5bbd-kube-api-access-2p29t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.635555 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.635459 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.635555 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.635485 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.736053 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.736026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.736174 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.736131 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p29t\" (UniqueName: \"kubernetes.io/projected/3c803c03-adfc-41bd-bc99-920f57cc5bbd-kube-api-access-2p29t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.736174 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.736166 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.736402 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.736379 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.736469 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.736426 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.744658 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.744636 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p29t\" (UniqueName: \"kubernetes.io/projected/3c803c03-adfc-41bd-bc99-920f57cc5bbd-kube-api-access-2p29t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:33.905010 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:33.904942 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:34.036975 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:34.036947 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd"] Jul 01 23:09:34.040104 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:34.040074 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c803c03_adfc_41bd_bc99_920f57cc5bbd.slice/crio-b3aaf759d466579bab3c4da619bcbc3551b1753a6facb0746ae5420a6a3ed4ec WatchSource:0}: Error finding container b3aaf759d466579bab3c4da619bcbc3551b1753a6facb0746ae5420a6a3ed4ec: Status 404 returned error can't find the container with id b3aaf759d466579bab3c4da619bcbc3551b1753a6facb0746ae5420a6a3ed4ec Jul 01 23:09:34.261131 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:34.261033 2569 generic.go:356] "Generic (PLEG): container finished" podID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerID="00b4510fe31900c4a25220666629db0a6c4a1610d93f90b6efc0e50f0769f92e" exitCode=0 Jul 01 23:09:34.261427 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:34.261122 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" event={"ID":"3c803c03-adfc-41bd-bc99-920f57cc5bbd","Type":"ContainerDied","Data":"00b4510fe31900c4a25220666629db0a6c4a1610d93f90b6efc0e50f0769f92e"} Jul 01 23:09:34.261427 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:34.261171 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" event={"ID":"3c803c03-adfc-41bd-bc99-920f57cc5bbd","Type":"ContainerStarted","Data":"b3aaf759d466579bab3c4da619bcbc3551b1753a6facb0746ae5420a6a3ed4ec"} Jul 01 23:09:34.262197 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:34.262182 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 23:09:36.275598 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:36.275569 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" event={"ID":"3c803c03-adfc-41bd-bc99-920f57cc5bbd","Type":"ContainerStarted","Data":"d20bebaf8764d3ac9742ff2ffd70d4009330bd70f172c2880582a4db274c1e8e"} Jul 01 23:09:37.281553 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:37.281522 2569 generic.go:356] "Generic (PLEG): container finished" podID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerID="d20bebaf8764d3ac9742ff2ffd70d4009330bd70f172c2880582a4db274c1e8e" exitCode=0 Jul 01 23:09:37.282015 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:37.281627 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" event={"ID":"3c803c03-adfc-41bd-bc99-920f57cc5bbd","Type":"ContainerDied","Data":"d20bebaf8764d3ac9742ff2ffd70d4009330bd70f172c2880582a4db274c1e8e"} Jul 01 23:09:38.113798 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.113768 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-8pttx"] Jul 01 23:09:38.116654 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.116637 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.119153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.119134 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 01 23:09:38.119153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.119142 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-27dl4\"" Jul 01 23:09:38.125151 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.125129 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 01 23:09:38.133912 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.133892 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-8pttx"] Jul 01 23:09:38.175491 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.175468 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjbzr\" (UniqueName: \"kubernetes.io/projected/1ca9fdaa-8057-41f0-a570-e08d420f1951-kube-api-access-gjbzr\") pod \"cert-manager-cainjector-5d99449ffd-8pttx\" (UID: \"1ca9fdaa-8057-41f0-a570-e08d420f1951\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.175594 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.175515 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ca9fdaa-8057-41f0-a570-e08d420f1951-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-8pttx\" (UID: \"1ca9fdaa-8057-41f0-a570-e08d420f1951\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.275994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.275965 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjbzr\" (UniqueName: \"kubernetes.io/projected/1ca9fdaa-8057-41f0-a570-e08d420f1951-kube-api-access-gjbzr\") pod \"cert-manager-cainjector-5d99449ffd-8pttx\" (UID: \"1ca9fdaa-8057-41f0-a570-e08d420f1951\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.276128 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.276026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ca9fdaa-8057-41f0-a570-e08d420f1951-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-8pttx\" (UID: \"1ca9fdaa-8057-41f0-a570-e08d420f1951\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.285452 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.285426 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ca9fdaa-8057-41f0-a570-e08d420f1951-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-8pttx\" (UID: \"1ca9fdaa-8057-41f0-a570-e08d420f1951\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.285763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.285522 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjbzr\" (UniqueName: \"kubernetes.io/projected/1ca9fdaa-8057-41f0-a570-e08d420f1951-kube-api-access-gjbzr\") pod \"cert-manager-cainjector-5d99449ffd-8pttx\" (UID: \"1ca9fdaa-8057-41f0-a570-e08d420f1951\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.288957 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.288936 2569 generic.go:356] "Generic (PLEG): container finished" podID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerID="91584a90e19ac9ad2bca79124157ef5180d266a61b9aad1fc6e80ffdf1d716b7" exitCode=0 Jul 01 23:09:38.289047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.289020 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" event={"ID":"3c803c03-adfc-41bd-bc99-920f57cc5bbd","Type":"ContainerDied","Data":"91584a90e19ac9ad2bca79124157ef5180d266a61b9aad1fc6e80ffdf1d716b7"} Jul 01 23:09:38.442484 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.442398 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" Jul 01 23:09:38.574578 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:38.574556 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-8pttx"] Jul 01 23:09:38.576758 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:38.576734 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ca9fdaa_8057_41f0_a570_e08d420f1951.slice/crio-58f305bda488e53a128908fbdc0a408f3bf4b2441275e1e715b8079dee86c62c WatchSource:0}: Error finding container 58f305bda488e53a128908fbdc0a408f3bf4b2441275e1e715b8079dee86c62c: Status 404 returned error can't find the container with id 58f305bda488e53a128908fbdc0a408f3bf4b2441275e1e715b8079dee86c62c Jul 01 23:09:39.295639 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.295589 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" event={"ID":"1ca9fdaa-8057-41f0-a570-e08d420f1951","Type":"ContainerStarted","Data":"58f305bda488e53a128908fbdc0a408f3bf4b2441275e1e715b8079dee86c62c"} Jul 01 23:09:39.441356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.441333 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:39.485697 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.485662 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p29t\" (UniqueName: \"kubernetes.io/projected/3c803c03-adfc-41bd-bc99-920f57cc5bbd-kube-api-access-2p29t\") pod \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " Jul 01 23:09:39.485885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.485715 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-bundle\") pod \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " Jul 01 23:09:39.485885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.485739 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-util\") pod \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\" (UID: \"3c803c03-adfc-41bd-bc99-920f57cc5bbd\") " Jul 01 23:09:39.486144 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.486120 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-bundle" (OuterVolumeSpecName: "bundle") pod "3c803c03-adfc-41bd-bc99-920f57cc5bbd" (UID: "3c803c03-adfc-41bd-bc99-920f57cc5bbd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:09:39.488031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.488003 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c803c03-adfc-41bd-bc99-920f57cc5bbd-kube-api-access-2p29t" (OuterVolumeSpecName: "kube-api-access-2p29t") pod "3c803c03-adfc-41bd-bc99-920f57cc5bbd" (UID: "3c803c03-adfc-41bd-bc99-920f57cc5bbd"). InnerVolumeSpecName "kube-api-access-2p29t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:09:39.492688 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.492658 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-util" (OuterVolumeSpecName: "util") pod "3c803c03-adfc-41bd-bc99-920f57cc5bbd" (UID: "3c803c03-adfc-41bd-bc99-920f57cc5bbd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:09:39.586641 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.586595 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p29t\" (UniqueName: \"kubernetes.io/projected/3c803c03-adfc-41bd-bc99-920f57cc5bbd-kube-api-access-2p29t\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:39.586641 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.586643 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:39.586836 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:39.586659 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c803c03-adfc-41bd-bc99-920f57cc5bbd-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:40.303331 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.303269 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" Jul 01 23:09:40.303331 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.303284 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frjlhd" event={"ID":"3c803c03-adfc-41bd-bc99-920f57cc5bbd","Type":"ContainerDied","Data":"b3aaf759d466579bab3c4da619bcbc3551b1753a6facb0746ae5420a6a3ed4ec"} Jul 01 23:09:40.303331 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.303336 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3aaf759d466579bab3c4da619bcbc3551b1753a6facb0746ae5420a6a3ed4ec" Jul 01 23:09:40.323189 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323155 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4"] Jul 01 23:09:40.323553 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323536 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="util" Jul 01 23:09:40.323673 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323556 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="util" Jul 01 23:09:40.323673 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323594 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="extract" Jul 01 23:09:40.323673 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323602 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="extract" Jul 01 23:09:40.323673 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323631 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="pull" Jul 01 23:09:40.323673 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323639 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="pull" Jul 01 23:09:40.323904 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.323726 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c803c03-adfc-41bd-bc99-920f57cc5bbd" containerName="extract" Jul 01 23:09:40.326868 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.326840 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.329451 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.329429 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2svpj\"" Jul 01 23:09:40.339647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.339567 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4"] Jul 01 23:09:40.393095 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.393058 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/755ed298-fc05-4c43-aab0-2a87be82dba2-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-hx6s4\" (UID: \"755ed298-fc05-4c43-aab0-2a87be82dba2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.393263 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.393119 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/755ed298-fc05-4c43-aab0-2a87be82dba2-kube-api-access-mlr7d\") pod \"cert-manager-webhook-7dc76cbb94-hx6s4\" (UID: \"755ed298-fc05-4c43-aab0-2a87be82dba2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.493618 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.493581 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/755ed298-fc05-4c43-aab0-2a87be82dba2-kube-api-access-mlr7d\") pod \"cert-manager-webhook-7dc76cbb94-hx6s4\" (UID: \"755ed298-fc05-4c43-aab0-2a87be82dba2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.493780 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.493717 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/755ed298-fc05-4c43-aab0-2a87be82dba2-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-hx6s4\" (UID: \"755ed298-fc05-4c43-aab0-2a87be82dba2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.502976 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.502949 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/755ed298-fc05-4c43-aab0-2a87be82dba2-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-hx6s4\" (UID: \"755ed298-fc05-4c43-aab0-2a87be82dba2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.503387 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.503363 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/755ed298-fc05-4c43-aab0-2a87be82dba2-kube-api-access-mlr7d\") pod \"cert-manager-webhook-7dc76cbb94-hx6s4\" (UID: \"755ed298-fc05-4c43-aab0-2a87be82dba2\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:40.637124 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:40.637084 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:41.151743 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.151713 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4"] Jul 01 23:09:41.154230 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:41.154202 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod755ed298_fc05_4c43_aab0_2a87be82dba2.slice/crio-3ab9bf69eb56b068f5856b1d48fc1b8433b9008ec16d8f7d8a812ad864a1554b WatchSource:0}: Error finding container 3ab9bf69eb56b068f5856b1d48fc1b8433b9008ec16d8f7d8a812ad864a1554b: Status 404 returned error can't find the container with id 3ab9bf69eb56b068f5856b1d48fc1b8433b9008ec16d8f7d8a812ad864a1554b Jul 01 23:09:41.310400 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.310319 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" event={"ID":"755ed298-fc05-4c43-aab0-2a87be82dba2","Type":"ContainerStarted","Data":"5470203a0a09dc0087c13778da40627657dedc85a066bc34b74e3774fe79d1f0"} Jul 01 23:09:41.310400 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.310363 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" event={"ID":"755ed298-fc05-4c43-aab0-2a87be82dba2","Type":"ContainerStarted","Data":"3ab9bf69eb56b068f5856b1d48fc1b8433b9008ec16d8f7d8a812ad864a1554b"} Jul 01 23:09:41.310857 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.310432 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:41.312144 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.312114 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" event={"ID":"1ca9fdaa-8057-41f0-a570-e08d420f1951","Type":"ContainerStarted","Data":"65d5e660b12e047081f409830e7c70143e56e87e4375e969cc07532537ef4a59"} Jul 01 23:09:41.330124 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.330075 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" podStartSLOduration=1.33005829 podStartE2EDuration="1.33005829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:09:41.329739653 +0000 UTC m=+311.552267866" watchObservedRunningTime="2026-07-01 23:09:41.33005829 +0000 UTC m=+311.552586501" Jul 01 23:09:41.348946 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:41.348900 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-8pttx" podStartSLOduration=0.835666425 podStartE2EDuration="3.348882392s" totalImagesPullingTime="2.513215967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:09:41.346099928 +0000 UTC m=+311.568628131" watchObservedRunningTime="2026-07-01 23:09:41.348882392 +0000 UTC m=+311.571410603" Jul 01 23:09:44.228920 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.228884 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-zrclm"] Jul 01 23:09:44.232178 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.232162 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.234659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.234638 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-vbkjg\"" Jul 01 23:09:44.243594 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.243574 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-zrclm"] Jul 01 23:09:44.328570 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.328542 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2181927f-3281-48be-9f9a-4e3a1b9d9aea-bound-sa-token\") pod \"cert-manager-59c77684cc-zrclm\" (UID: \"2181927f-3281-48be-9f9a-4e3a1b9d9aea\") " pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.328764 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.328633 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkcfj\" (UniqueName: \"kubernetes.io/projected/2181927f-3281-48be-9f9a-4e3a1b9d9aea-kube-api-access-bkcfj\") pod \"cert-manager-59c77684cc-zrclm\" (UID: \"2181927f-3281-48be-9f9a-4e3a1b9d9aea\") " pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.429724 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.429698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2181927f-3281-48be-9f9a-4e3a1b9d9aea-bound-sa-token\") pod \"cert-manager-59c77684cc-zrclm\" (UID: \"2181927f-3281-48be-9f9a-4e3a1b9d9aea\") " pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.429873 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.429802 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkcfj\" (UniqueName: \"kubernetes.io/projected/2181927f-3281-48be-9f9a-4e3a1b9d9aea-kube-api-access-bkcfj\") pod \"cert-manager-59c77684cc-zrclm\" (UID: \"2181927f-3281-48be-9f9a-4e3a1b9d9aea\") " pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.438051 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.438029 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2181927f-3281-48be-9f9a-4e3a1b9d9aea-bound-sa-token\") pod \"cert-manager-59c77684cc-zrclm\" (UID: \"2181927f-3281-48be-9f9a-4e3a1b9d9aea\") " pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.438178 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.438166 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkcfj\" (UniqueName: \"kubernetes.io/projected/2181927f-3281-48be-9f9a-4e3a1b9d9aea-kube-api-access-bkcfj\") pod \"cert-manager-59c77684cc-zrclm\" (UID: \"2181927f-3281-48be-9f9a-4e3a1b9d9aea\") " pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.541550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.541470 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-zrclm" Jul 01 23:09:44.683138 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:44.683053 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-zrclm"] Jul 01 23:09:44.688842 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:44.688808 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2181927f_3281_48be_9f9a_4e3a1b9d9aea.slice/crio-9c3efaf23e75791a7ebeafead6131997fcb025206a49c59b515f2a3ffc99774e WatchSource:0}: Error finding container 9c3efaf23e75791a7ebeafead6131997fcb025206a49c59b515f2a3ffc99774e: Status 404 returned error can't find the container with id 9c3efaf23e75791a7ebeafead6131997fcb025206a49c59b515f2a3ffc99774e Jul 01 23:09:45.336837 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:45.336789 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-zrclm" event={"ID":"2181927f-3281-48be-9f9a-4e3a1b9d9aea","Type":"ContainerStarted","Data":"d6c7cb15f682425992967aaaf6e58625b82d1e23d6057ca52ca1be350211d2ca"} Jul 01 23:09:45.336837 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:45.336838 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-zrclm" event={"ID":"2181927f-3281-48be-9f9a-4e3a1b9d9aea","Type":"ContainerStarted","Data":"9c3efaf23e75791a7ebeafead6131997fcb025206a49c59b515f2a3ffc99774e"} Jul 01 23:09:45.355034 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:45.354988 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-zrclm" podStartSLOduration=1.354973556 podStartE2EDuration="1.354973556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:09:45.352993458 +0000 UTC m=+315.575521681" watchObservedRunningTime="2026-07-01 23:09:45.354973556 +0000 UTC m=+315.577501767" Jul 01 23:09:47.323687 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:47.323651 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-hx6s4" Jul 01 23:09:54.643182 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.643148 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb"] Jul 01 23:09:54.665063 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.665033 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb"] Jul 01 23:09:54.665215 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.665156 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.667783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.667759 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 01 23:09:54.667925 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.667761 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 01 23:09:54.668483 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.668461 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6j5mm\"" Jul 01 23:09:54.714470 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.714432 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.714656 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.714487 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.714656 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.714517 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6nzp\" (UniqueName: \"kubernetes.io/projected/b24aaebe-3f9b-49a7-8403-fe78727e844e-kube-api-access-r6nzp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.815425 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.815384 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.815585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.815447 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.815585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.815468 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6nzp\" (UniqueName: \"kubernetes.io/projected/b24aaebe-3f9b-49a7-8403-fe78727e844e-kube-api-access-r6nzp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.815827 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.815807 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.815863 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.815832 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.825066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.825034 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6nzp\" (UniqueName: \"kubernetes.io/projected/b24aaebe-3f9b-49a7-8403-fe78727e844e-kube-api-access-r6nzp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:54.974890 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:54.974819 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:55.120185 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:55.120144 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb"] Jul 01 23:09:55.122506 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:09:55.122482 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24aaebe_3f9b_49a7_8403_fe78727e844e.slice/crio-a82c3d91294a5b052f2d2f3bc5e90c539243642607cd50b4a504006dcab6decb WatchSource:0}: Error finding container a82c3d91294a5b052f2d2f3bc5e90c539243642607cd50b4a504006dcab6decb: Status 404 returned error can't find the container with id a82c3d91294a5b052f2d2f3bc5e90c539243642607cd50b4a504006dcab6decb Jul 01 23:09:55.382656 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:55.382624 2569 generic.go:356] "Generic (PLEG): container finished" podID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerID="79decce673fe41aa244270f94725b256481373cb958a71552faeb4ced60a3c1c" exitCode=0 Jul 01 23:09:55.382810 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:55.382717 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" event={"ID":"b24aaebe-3f9b-49a7-8403-fe78727e844e","Type":"ContainerDied","Data":"79decce673fe41aa244270f94725b256481373cb958a71552faeb4ced60a3c1c"} Jul 01 23:09:55.382810 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:55.382770 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" event={"ID":"b24aaebe-3f9b-49a7-8403-fe78727e844e","Type":"ContainerStarted","Data":"a82c3d91294a5b052f2d2f3bc5e90c539243642607cd50b4a504006dcab6decb"} Jul 01 23:09:56.389892 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:56.389810 2569 generic.go:356] "Generic (PLEG): container finished" podID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerID="5bf808d86674037af6dd17ce42151a2143c53b0edbb72561fbb91e0ba19e14bf" exitCode=0 Jul 01 23:09:56.390227 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:56.389897 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" event={"ID":"b24aaebe-3f9b-49a7-8403-fe78727e844e","Type":"ContainerDied","Data":"5bf808d86674037af6dd17ce42151a2143c53b0edbb72561fbb91e0ba19e14bf"} Jul 01 23:09:57.396077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:57.396047 2569 generic.go:356] "Generic (PLEG): container finished" podID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerID="ad18af4d9011ba08d330e8a0abe755d6b0b5448a1bf54df64f68583af3f2e6a5" exitCode=0 Jul 01 23:09:57.396444 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:57.396082 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" event={"ID":"b24aaebe-3f9b-49a7-8403-fe78727e844e","Type":"ContainerDied","Data":"ad18af4d9011ba08d330e8a0abe755d6b0b5448a1bf54df64f68583af3f2e6a5"} Jul 01 23:09:58.524766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.524745 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:09:58.650111 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.650080 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-bundle\") pod \"b24aaebe-3f9b-49a7-8403-fe78727e844e\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " Jul 01 23:09:58.650241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.650133 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-util\") pod \"b24aaebe-3f9b-49a7-8403-fe78727e844e\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " Jul 01 23:09:58.650241 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.650216 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6nzp\" (UniqueName: \"kubernetes.io/projected/b24aaebe-3f9b-49a7-8403-fe78727e844e-kube-api-access-r6nzp\") pod \"b24aaebe-3f9b-49a7-8403-fe78727e844e\" (UID: \"b24aaebe-3f9b-49a7-8403-fe78727e844e\") " Jul 01 23:09:58.650901 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.650840 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-bundle" (OuterVolumeSpecName: "bundle") pod "b24aaebe-3f9b-49a7-8403-fe78727e844e" (UID: "b24aaebe-3f9b-49a7-8403-fe78727e844e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:09:58.652329 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.652301 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b24aaebe-3f9b-49a7-8403-fe78727e844e-kube-api-access-r6nzp" (OuterVolumeSpecName: "kube-api-access-r6nzp") pod "b24aaebe-3f9b-49a7-8403-fe78727e844e" (UID: "b24aaebe-3f9b-49a7-8403-fe78727e844e"). InnerVolumeSpecName "kube-api-access-r6nzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:09:58.656163 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.656140 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-util" (OuterVolumeSpecName: "util") pod "b24aaebe-3f9b-49a7-8403-fe78727e844e" (UID: "b24aaebe-3f9b-49a7-8403-fe78727e844e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:09:58.751501 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.751471 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6nzp\" (UniqueName: \"kubernetes.io/projected/b24aaebe-3f9b-49a7-8403-fe78727e844e-kube-api-access-r6nzp\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:58.751501 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.751496 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:58.751501 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:58.751504 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b24aaebe-3f9b-49a7-8403-fe78727e844e-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:09:59.407090 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:59.407058 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" event={"ID":"b24aaebe-3f9b-49a7-8403-fe78727e844e","Type":"ContainerDied","Data":"a82c3d91294a5b052f2d2f3bc5e90c539243642607cd50b4a504006dcab6decb"} Jul 01 23:09:59.407363 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:59.407105 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a82c3d91294a5b052f2d2f3bc5e90c539243642607cd50b4a504006dcab6decb" Jul 01 23:09:59.407363 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:09:59.407068 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzrrqb" Jul 01 23:10:07.127058 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127029 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj"] Jul 01 23:10:07.127442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127364 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="util" Jul 01 23:10:07.127442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127376 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="util" Jul 01 23:10:07.127442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127393 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="extract" Jul 01 23:10:07.127442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127399 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="extract" Jul 01 23:10:07.127442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127419 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="pull" Jul 01 23:10:07.127442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127425 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="pull" Jul 01 23:10:07.127697 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.127483 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b24aaebe-3f9b-49a7-8403-fe78727e844e" containerName="extract" Jul 01 23:10:07.132393 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.132376 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.135710 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.135688 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 01 23:10:07.136847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.136825 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 01 23:10:07.136951 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.136905 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 01 23:10:07.137013 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.136980 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zbwjr\"" Jul 01 23:10:07.137523 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.137507 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 01 23:10:07.157823 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.157799 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj"] Jul 01 23:10:07.218899 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.218869 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266a292b-1ade-4700-93c3-fcedebf0b378-tmp\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.219066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.218903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266a292b-1ade-4700-93c3-fcedebf0b378-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.219066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.218928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8g9j\" (UniqueName: \"kubernetes.io/projected/266a292b-1ade-4700-93c3-fcedebf0b378-kube-api-access-v8g9j\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.219066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.218996 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/266a292b-1ade-4700-93c3-fcedebf0b378-manifests\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.219066 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.219054 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/266a292b-1ade-4700-93c3-fcedebf0b378-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.319877 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.319840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/266a292b-1ade-4700-93c3-fcedebf0b378-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.320071 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.319899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266a292b-1ade-4700-93c3-fcedebf0b378-tmp\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.320071 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.319918 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266a292b-1ade-4700-93c3-fcedebf0b378-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.320071 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.319939 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8g9j\" (UniqueName: \"kubernetes.io/projected/266a292b-1ade-4700-93c3-fcedebf0b378-kube-api-access-v8g9j\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.320071 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.319968 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/266a292b-1ade-4700-93c3-fcedebf0b378-manifests\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.320459 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.320433 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/266a292b-1ade-4700-93c3-fcedebf0b378-manifests\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.322175 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.322147 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/266a292b-1ade-4700-93c3-fcedebf0b378-tmp\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.322708 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.322683 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266a292b-1ade-4700-93c3-fcedebf0b378-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.323008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.322987 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/266a292b-1ade-4700-93c3-fcedebf0b378-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.330752 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.330727 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8g9j\" (UniqueName: \"kubernetes.io/projected/266a292b-1ade-4700-93c3-fcedebf0b378-kube-api-access-v8g9j\") pod \"opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj\" (UID: \"266a292b-1ade-4700-93c3-fcedebf0b378\") " pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.442715 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.442602 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:07.583880 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:07.583844 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj"] Jul 01 23:10:07.588702 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:07.588664 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266a292b_1ade_4700_93c3_fcedebf0b378.slice/crio-7074322c57b7988ceeb12b7539500d3dd866ca53fde5c22f9bba2195ebe777dc WatchSource:0}: Error finding container 7074322c57b7988ceeb12b7539500d3dd866ca53fde5c22f9bba2195ebe777dc: Status 404 returned error can't find the container with id 7074322c57b7988ceeb12b7539500d3dd866ca53fde5c22f9bba2195ebe777dc Jul 01 23:10:08.458794 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:08.458748 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" event={"ID":"266a292b-1ade-4700-93c3-fcedebf0b378","Type":"ContainerStarted","Data":"7074322c57b7988ceeb12b7539500d3dd866ca53fde5c22f9bba2195ebe777dc"} Jul 01 23:10:11.479947 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:11.479914 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" event={"ID":"266a292b-1ade-4700-93c3-fcedebf0b378","Type":"ContainerStarted","Data":"b5272dc971559e842b9cdb0c731100390d760970028111b9ec491cf665493510"} Jul 01 23:10:12.486011 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:12.485982 2569 generic.go:356] "Generic (PLEG): container finished" podID="266a292b-1ade-4700-93c3-fcedebf0b378" containerID="b5272dc971559e842b9cdb0c731100390d760970028111b9ec491cf665493510" exitCode=0 Jul 01 23:10:12.486449 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:12.486075 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" event={"ID":"266a292b-1ade-4700-93c3-fcedebf0b378","Type":"ContainerDied","Data":"b5272dc971559e842b9cdb0c731100390d760970028111b9ec491cf665493510"} Jul 01 23:10:13.493663 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:13.493622 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" event={"ID":"266a292b-1ade-4700-93c3-fcedebf0b378","Type":"ContainerStarted","Data":"738703999aad6c96a40c219abe0ec2e3db7c1b3f72802395b56af39aac55a420"} Jul 01 23:10:13.494145 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:13.493750 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:13.517374 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:13.517328 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" podStartSLOduration=2.123465876 podStartE2EDuration="6.517312795s" totalImagesPullingTime="3.328772223s" totalInitContainerRuntime="1.065074696s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:13.51544557 +0000 UTC m=+343.737973782" watchObservedRunningTime="2026-07-01 23:10:13.517312795 +0000 UTC m=+343.739841004" Jul 01 23:10:24.500455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:24.500373 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj" Jul 01 23:10:27.889453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.889420 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-869858444d-xzcrk"] Jul 01 23:10:27.894180 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.894158 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:27.896775 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.896750 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 01 23:10:27.897109 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.897092 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 01 23:10:27.897214 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.897199 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 01 23:10:27.897581 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.897569 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 01 23:10:27.899679 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.899659 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9cd41c9-e389-4786-9656-54be4100f511-tls-certs\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:27.899766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.899718 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9cd41c9-e389-4786-9656-54be4100f511-tmp\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:27.899766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.899744 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfhbx\" (UniqueName: \"kubernetes.io/projected/a9cd41c9-e389-4786-9656-54be4100f511-kube-api-access-dfhbx\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:27.912565 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:27.912544 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869858444d-xzcrk"] Jul 01 23:10:28.000323 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.000289 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9cd41c9-e389-4786-9656-54be4100f511-tmp\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.000323 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.000329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfhbx\" (UniqueName: \"kubernetes.io/projected/a9cd41c9-e389-4786-9656-54be4100f511-kube-api-access-dfhbx\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.000589 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.000381 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9cd41c9-e389-4786-9656-54be4100f511-tls-certs\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.002737 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.002711 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9cd41c9-e389-4786-9656-54be4100f511-tmp\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.003022 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.003001 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9cd41c9-e389-4786-9656-54be4100f511-tls-certs\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.019280 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.019250 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfhbx\" (UniqueName: \"kubernetes.io/projected/a9cd41c9-e389-4786-9656-54be4100f511-kube-api-access-dfhbx\") pod \"kube-auth-proxy-869858444d-xzcrk\" (UID: \"a9cd41c9-e389-4786-9656-54be4100f511\") " pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.159462 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.159389 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mq9fx"] Jul 01 23:10:28.162993 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.162977 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.166650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.166594 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 01 23:10:28.166805 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.166790 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ql7df\"" Jul 01 23:10:28.174115 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.174080 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mq9fx"] Jul 01 23:10:28.202363 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.202330 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sf6g\" (UniqueName: \"kubernetes.io/projected/0cde6afd-ff4d-4720-a076-e937faf27b99-kube-api-access-8sf6g\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.202553 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.202371 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cde6afd-ff4d-4720-a076-e937faf27b99-cert\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.203162 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.203131 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" Jul 01 23:10:28.303393 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.303363 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sf6g\" (UniqueName: \"kubernetes.io/projected/0cde6afd-ff4d-4720-a076-e937faf27b99-kube-api-access-8sf6g\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.303556 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.303409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cde6afd-ff4d-4720-a076-e937faf27b99-cert\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.303656 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:10:28.303581 2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 01 23:10:28.303714 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:10:28.303673 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cde6afd-ff4d-4720-a076-e937faf27b99-cert podName:0cde6afd-ff4d-4720-a076-e937faf27b99 nodeName:}" failed. No retries permitted until 2026-07-01 23:10:28.803650179 +0000 UTC m=+359.026178370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0cde6afd-ff4d-4720-a076-e937faf27b99-cert") pod "odh-model-controller-7998c67dc4-mq9fx" (UID: "0cde6afd-ff4d-4720-a076-e937faf27b99") : secret "odh-model-controller-webhook-cert" not found Jul 01 23:10:28.320804 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.319564 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sf6g\" (UniqueName: \"kubernetes.io/projected/0cde6afd-ff4d-4720-a076-e937faf27b99-kube-api-access-8sf6g\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.351709 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.351625 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869858444d-xzcrk"] Jul 01 23:10:28.353554 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:28.353512 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9cd41c9_e389_4786_9656_54be4100f511.slice/crio-82355edaf0e0899b997a2d3ebe215f9c893c0ce69bea86fdb77b5638814868a4 WatchSource:0}: Error finding container 82355edaf0e0899b997a2d3ebe215f9c893c0ce69bea86fdb77b5638814868a4: Status 404 returned error can't find the container with id 82355edaf0e0899b997a2d3ebe215f9c893c0ce69bea86fdb77b5638814868a4 Jul 01 23:10:28.570551 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.570519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" event={"ID":"a9cd41c9-e389-4786-9656-54be4100f511","Type":"ContainerStarted","Data":"82355edaf0e0899b997a2d3ebe215f9c893c0ce69bea86fdb77b5638814868a4"} Jul 01 23:10:28.807540 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.807509 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cde6afd-ff4d-4720-a076-e937faf27b99-cert\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:28.810563 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:28.810532 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cde6afd-ff4d-4720-a076-e937faf27b99-cert\") pod \"odh-model-controller-7998c67dc4-mq9fx\" (UID: \"0cde6afd-ff4d-4720-a076-e937faf27b99\") " pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:29.076004 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.075937 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:29.162957 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.162928 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6rb82"] Jul 01 23:10:29.167222 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.166985 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.170955 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.170867 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 01 23:10:29.188834 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.188809 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6rb82"] Jul 01 23:10:29.261751 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:29.261702 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cde6afd_ff4d_4720_a076_e937faf27b99.slice/crio-b6d273088cdc12e4275d27d22980863ead99c690026db1bbe2ffe37a3f1160d0 WatchSource:0}: Error finding container b6d273088cdc12e4275d27d22980863ead99c690026db1bbe2ffe37a3f1160d0: Status 404 returned error can't find the container with id b6d273088cdc12e4275d27d22980863ead99c690026db1bbe2ffe37a3f1160d0 Jul 01 23:10:29.262219 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.262188 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mq9fx"] Jul 01 23:10:29.311950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.311906 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d5c636-734f-490d-a0c1-348c9727dff6-tls-certs\") pod \"model-serving-api-54d89bd5fc-6rb82\" (UID: \"19d5c636-734f-490d-a0c1-348c9727dff6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.312135 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.312061 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz6ps\" (UniqueName: \"kubernetes.io/projected/19d5c636-734f-490d-a0c1-348c9727dff6-kube-api-access-tz6ps\") pod \"model-serving-api-54d89bd5fc-6rb82\" (UID: \"19d5c636-734f-490d-a0c1-348c9727dff6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.358531 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.358460 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f"] Jul 01 23:10:29.362252 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.362230 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.365409 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.365370 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 01 23:10:29.365532 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.365512 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 01 23:10:29.365907 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.365772 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9hzr9\"" Jul 01 23:10:29.365907 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.365806 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 01 23:10:29.366343 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.366294 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 01 23:10:29.380949 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.380927 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f"] Jul 01 23:10:29.413141 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413112 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz6ps\" (UniqueName: \"kubernetes.io/projected/19d5c636-734f-490d-a0c1-348c9727dff6-kube-api-access-tz6ps\") pod \"model-serving-api-54d89bd5fc-6rb82\" (UID: \"19d5c636-734f-490d-a0c1-348c9727dff6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.413287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413151 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-token\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.413287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413175 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-cacerts\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.413287 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413210 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.413414 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413295 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.413414 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413332 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.413414 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413366 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d5c636-734f-490d-a0c1-348c9727dff6-tls-certs\") pod \"model-serving-api-54d89bd5fc-6rb82\" (UID: \"19d5c636-734f-490d-a0c1-348c9727dff6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.413560 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413418 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4aa64a-395a-436c-b3e3-00120af9b1fd-local-certs\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.413560 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.413487 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwgvd\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-kube-api-access-hwgvd\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.415753 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.415735 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d5c636-734f-490d-a0c1-348c9727dff6-tls-certs\") pod \"model-serving-api-54d89bd5fc-6rb82\" (UID: \"19d5c636-734f-490d-a0c1-348c9727dff6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.428986 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.428960 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz6ps\" (UniqueName: \"kubernetes.io/projected/19d5c636-734f-490d-a0c1-348c9727dff6-kube-api-access-tz6ps\") pod \"model-serving-api-54d89bd5fc-6rb82\" (UID: \"19d5c636-734f-490d-a0c1-348c9727dff6\") " pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.491258 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.491224 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:29.514084 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.514242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514099 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.514242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4aa64a-395a-436c-b3e3-00120af9b1fd-local-certs\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.514242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514160 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwgvd\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-kube-api-access-hwgvd\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.514242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-token\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.514453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514247 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-cacerts\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.514453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.514273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.515449 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.515384 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.517654 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.517272 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.517826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.517802 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.518945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.518894 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-cacerts\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.519538 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.519245 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4aa64a-395a-436c-b3e3-00120af9b1fd-local-certs\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.523506 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.523463 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwgvd\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-kube-api-access-hwgvd\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.523929 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.523906 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-token\") pod \"istiod-openshift-gateway-7574459656-xrq6f\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.578847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.578777 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" event={"ID":"0cde6afd-ff4d-4720-a076-e937faf27b99","Type":"ContainerStarted","Data":"b6d273088cdc12e4275d27d22980863ead99c690026db1bbe2ffe37a3f1160d0"} Jul 01 23:10:29.667235 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.666364 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6rb82"] Jul 01 23:10:29.669845 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:29.669805 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d5c636_734f_490d_a0c1_348c9727dff6.slice/crio-14261739ce34280451cf3294294b277f8566ca686317ead10cb06ffd7da85ed2 WatchSource:0}: Error finding container 14261739ce34280451cf3294294b277f8566ca686317ead10cb06ffd7da85ed2: Status 404 returned error can't find the container with id 14261739ce34280451cf3294294b277f8566ca686317ead10cb06ffd7da85ed2 Jul 01 23:10:29.676309 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.676125 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:29.876905 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:29.876811 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f"] Jul 01 23:10:29.880003 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:29.879969 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4aa64a_395a_436c_b3e3_00120af9b1fd.slice/crio-c53cb3a76119e4d0e36e01c381a181d530da9596c6701566edb2b52027af3670 WatchSource:0}: Error finding container c53cb3a76119e4d0e36e01c381a181d530da9596c6701566edb2b52027af3670: Status 404 returned error can't find the container with id c53cb3a76119e4d0e36e01c381a181d530da9596c6701566edb2b52027af3670 Jul 01 23:10:30.588730 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:30.588640 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" event={"ID":"19d5c636-734f-490d-a0c1-348c9727dff6","Type":"ContainerStarted","Data":"14261739ce34280451cf3294294b277f8566ca686317ead10cb06ffd7da85ed2"} Jul 01 23:10:30.591065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:30.591003 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" event={"ID":"4c4aa64a-395a-436c-b3e3-00120af9b1fd","Type":"ContainerStarted","Data":"c53cb3a76119e4d0e36e01c381a181d530da9596c6701566edb2b52027af3670"} Jul 01 23:10:33.736080 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:33.736059 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 01 23:10:34.618961 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.618926 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" event={"ID":"0cde6afd-ff4d-4720-a076-e937faf27b99","Type":"ContainerStarted","Data":"ba84423232a5d95e8f5c7afffbde36158a09d2c5edb46ac1e98e86f25d33c34a"} Jul 01 23:10:34.619153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.618990 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:34.620681 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.620646 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" event={"ID":"a9cd41c9-e389-4786-9656-54be4100f511","Type":"ContainerStarted","Data":"5330fdc15b2ecd2201799cdba5f76a063f630ca079b230bb6e1004c438762099"} Jul 01 23:10:34.622292 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.622270 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" event={"ID":"19d5c636-734f-490d-a0c1-348c9727dff6","Type":"ContainerStarted","Data":"1dd51b01d81915e6614c6585cd4a1c9a0156420d1c07394fb43c5e861e483bf9"} Jul 01 23:10:34.622431 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.622411 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:34.639319 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.639274 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" podStartSLOduration=2.116019942 podStartE2EDuration="6.639259346s" totalImagesPullingTime="4.523239404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:34.637507857 +0000 UTC m=+364.860036069" watchObservedRunningTime="2026-07-01 23:10:34.639259346 +0000 UTC m=+364.861787555" Jul 01 23:10:34.655247 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.655202 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" podStartSLOduration=1.593548126 podStartE2EDuration="5.655185303s" totalImagesPullingTime="4.061637177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:34.65442666 +0000 UTC m=+364.876954900" watchObservedRunningTime="2026-07-01 23:10:34.655185303 +0000 UTC m=+364.877713514" Jul 01 23:10:34.676675 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:34.676620 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-869858444d-xzcrk" podStartSLOduration=2.298363772 podStartE2EDuration="7.67658435s" totalImagesPullingTime="5.378220578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:34.673932736 +0000 UTC m=+364.896460948" watchObservedRunningTime="2026-07-01 23:10:34.67658435 +0000 UTC m=+364.899112561" Jul 01 23:10:35.424171 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:35.424134 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:10:35.424430 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:35.424196 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:10:35.629705 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:35.629660 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" event={"ID":"4c4aa64a-395a-436c-b3e3-00120af9b1fd","Type":"ContainerStarted","Data":"278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d"} Jul 01 23:10:35.630198 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:35.630176 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:35.677834 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:35.677718 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" podStartSLOduration=1.136826763 podStartE2EDuration="6.677698128s" totalImagesPullingTime="5.540871365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:35.664280979 +0000 UTC m=+365.886809226" watchObservedRunningTime="2026-07-01 23:10:35.677698128 +0000 UTC m=+365.900226342" Jul 01 23:10:36.636512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:36.636475 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-xrq6f container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 01 23:10:36.636898 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:36.636524 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" podUID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 23:10:37.640799 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:37.640768 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:40.447562 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.447526 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gn76v"] Jul 01 23:10:40.451498 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.451477 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:40.455541 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.455520 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 01 23:10:40.456947 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.456928 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7vgnd\"" Jul 01 23:10:40.468973 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.468953 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5"] Jul 01 23:10:40.472226 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.472210 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.476406 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.476388 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-f466j\"" Jul 01 23:10:40.476700 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.476679 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 01 23:10:40.479932 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.479915 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gn76v"] Jul 01 23:10:40.484858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.484841 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5"] Jul 01 23:10:40.518348 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.518325 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/152fc74c-3a9c-40ec-ba5a-9d94c7848149-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rfhb5\" (UID: \"152fc74c-3a9c-40ec-ba5a-9d94c7848149\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.518476 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.518363 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpgxv\" (UniqueName: \"kubernetes.io/projected/152fc74c-3a9c-40ec-ba5a-9d94c7848149-kube-api-access-vpgxv\") pod \"llmisvc-controller-manager-777c6bd7d5-rfhb5\" (UID: \"152fc74c-3a9c-40ec-ba5a-9d94c7848149\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.518476 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.518379 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0416fbc7-e700-4864-aa64-10b524e14ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:40.518476 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.518408 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw4qj\" (UniqueName: \"kubernetes.io/projected/0416fbc7-e700-4864-aa64-10b524e14ff9-kube-api-access-xw4qj\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:40.619934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.619879 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4qj\" (UniqueName: \"kubernetes.io/projected/0416fbc7-e700-4864-aa64-10b524e14ff9-kube-api-access-xw4qj\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:40.620119 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.620011 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/152fc74c-3a9c-40ec-ba5a-9d94c7848149-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rfhb5\" (UID: \"152fc74c-3a9c-40ec-ba5a-9d94c7848149\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.620119 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.620052 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpgxv\" (UniqueName: \"kubernetes.io/projected/152fc74c-3a9c-40ec-ba5a-9d94c7848149-kube-api-access-vpgxv\") pod \"llmisvc-controller-manager-777c6bd7d5-rfhb5\" (UID: \"152fc74c-3a9c-40ec-ba5a-9d94c7848149\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.620119 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.620076 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0416fbc7-e700-4864-aa64-10b524e14ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:40.620306 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:10:40.620230 2569 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 01 23:10:40.620306 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:10:40.620302 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0416fbc7-e700-4864-aa64-10b524e14ff9-cert podName:0416fbc7-e700-4864-aa64-10b524e14ff9 nodeName:}" failed. No retries permitted until 2026-07-01 23:10:41.120283105 +0000 UTC m=+371.342811303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0416fbc7-e700-4864-aa64-10b524e14ff9-cert") pod "kserve-controller-manager-d6f4b65b8-gn76v" (UID: "0416fbc7-e700-4864-aa64-10b524e14ff9") : secret "kserve-webhook-server-cert" not found Jul 01 23:10:40.622562 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.622538 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/152fc74c-3a9c-40ec-ba5a-9d94c7848149-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rfhb5\" (UID: \"152fc74c-3a9c-40ec-ba5a-9d94c7848149\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.631248 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.631227 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw4qj\" (UniqueName: \"kubernetes.io/projected/0416fbc7-e700-4864-aa64-10b524e14ff9-kube-api-access-xw4qj\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:40.632284 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.632264 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpgxv\" (UniqueName: \"kubernetes.io/projected/152fc74c-3a9c-40ec-ba5a-9d94c7848149-kube-api-access-vpgxv\") pod \"llmisvc-controller-manager-777c6bd7d5-rfhb5\" (UID: \"152fc74c-3a9c-40ec-ba5a-9d94c7848149\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.782649 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.782538 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:40.937574 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:40.937550 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5"] Jul 01 23:10:40.938997 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:40.938973 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod152fc74c_3a9c_40ec_ba5a_9d94c7848149.slice/crio-8bca0f038696fc6c8fe5533338003c5c88f21dd4ec96b869e7e8679208d38ab4 WatchSource:0}: Error finding container 8bca0f038696fc6c8fe5533338003c5c88f21dd4ec96b869e7e8679208d38ab4: Status 404 returned error can't find the container with id 8bca0f038696fc6c8fe5533338003c5c88f21dd4ec96b869e7e8679208d38ab4 Jul 01 23:10:41.123549 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:41.123515 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0416fbc7-e700-4864-aa64-10b524e14ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:41.125847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:41.125828 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0416fbc7-e700-4864-aa64-10b524e14ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-gn76v\" (UID: \"0416fbc7-e700-4864-aa64-10b524e14ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:41.361744 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:41.361715 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:41.521928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:41.521895 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gn76v"] Jul 01 23:10:41.522566 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:10:41.522542 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0416fbc7_e700_4864_aa64_10b524e14ff9.slice/crio-061f362d02353c5548e870d688848124a01794ddb3b2f5ca24cf54b634700704 WatchSource:0}: Error finding container 061f362d02353c5548e870d688848124a01794ddb3b2f5ca24cf54b634700704: Status 404 returned error can't find the container with id 061f362d02353c5548e870d688848124a01794ddb3b2f5ca24cf54b634700704 Jul 01 23:10:41.675828 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:41.675679 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" event={"ID":"0416fbc7-e700-4864-aa64-10b524e14ff9","Type":"ContainerStarted","Data":"061f362d02353c5548e870d688848124a01794ddb3b2f5ca24cf54b634700704"} Jul 01 23:10:41.677922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:41.677860 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" event={"ID":"152fc74c-3a9c-40ec-ba5a-9d94c7848149","Type":"ContainerStarted","Data":"8bca0f038696fc6c8fe5533338003c5c88f21dd4ec96b869e7e8679208d38ab4"} Jul 01 23:10:42.841970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:42.841800 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f"] Jul 01 23:10:42.842414 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:42.842121 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" podUID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" containerName="discovery" containerID="cri-o://278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d" gracePeriod=30 Jul 01 23:10:43.118867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.118845 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:43.244686 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244648 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-dns-cert\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.244904 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244716 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-token\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.244904 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244749 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-ca-configmap\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.244904 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244799 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4aa64a-395a-436c-b3e3-00120af9b1fd-local-certs\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.244904 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244826 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwgvd\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-kube-api-access-hwgvd\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.245108 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244910 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-cacerts\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.245108 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.244952 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-kubeconfig\") pod \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\" (UID: \"4c4aa64a-395a-436c-b3e3-00120af9b1fd\") " Jul 01 23:10:43.245215 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.245174 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:10:43.245314 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.245293 2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-ca-configmap\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.247780 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.247728 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4aa64a-395a-436c-b3e3-00120af9b1fd-local-certs" (OuterVolumeSpecName: "local-certs") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:10:43.248065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.248008 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:10:43.248065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.248042 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-cacerts" (OuterVolumeSpecName: "cacerts") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:10:43.248065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.248050 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-kube-api-access-hwgvd" (OuterVolumeSpecName: "kube-api-access-hwgvd") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "kube-api-access-hwgvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:10:43.248314 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.248255 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:10:43.248563 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.248542 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c4aa64a-395a-436c-b3e3-00120af9b1fd" (UID: "4c4aa64a-395a-436c-b3e3-00120af9b1fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:10:43.345742 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.345712 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-token\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.345742 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.345737 2569 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4aa64a-395a-436c-b3e3-00120af9b1fd-local-certs\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.345742 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.345746 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwgvd\" (UniqueName: \"kubernetes.io/projected/4c4aa64a-395a-436c-b3e3-00120af9b1fd-kube-api-access-hwgvd\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.345948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.345755 2569 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-cacerts\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.345948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.345764 2569 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-kubeconfig\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.345948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.345773 2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4c4aa64a-395a-436c-b3e3-00120af9b1fd-istio-csr-dns-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:10:43.691642 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.691587 2569 generic.go:356] "Generic (PLEG): container finished" podID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" containerID="278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d" exitCode=0 Jul 01 23:10:43.691783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.691653 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" event={"ID":"4c4aa64a-395a-436c-b3e3-00120af9b1fd","Type":"ContainerDied","Data":"278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d"} Jul 01 23:10:43.691783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.691688 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" event={"ID":"4c4aa64a-395a-436c-b3e3-00120af9b1fd","Type":"ContainerDied","Data":"c53cb3a76119e4d0e36e01c381a181d530da9596c6701566edb2b52027af3670"} Jul 01 23:10:43.691783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.691707 2569 scope.go:117] "RemoveContainer" containerID="278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d" Jul 01 23:10:43.691783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.691655 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f" Jul 01 23:10:43.700940 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.700918 2569 scope.go:117] "RemoveContainer" containerID="278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d" Jul 01 23:10:43.701177 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:10:43.701162 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d\": container with ID starting with 278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d not found: ID does not exist" containerID="278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d" Jul 01 23:10:43.701226 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.701183 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d"} err="failed to get container status \"278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d\": rpc error: code = NotFound desc = could not find container \"278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d\": container with ID starting with 278a08f664056da94ccde0e31a93cadbfa42afbcebdf4c49c67018154d4a6a1d not found: ID does not exist" Jul 01 23:10:43.715698 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.715673 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f"] Jul 01 23:10:43.721168 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:43.721136 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-xrq6f"] Jul 01 23:10:44.923601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:44.923579 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" path="/var/lib/kubelet/pods/4c4aa64a-395a-436c-b3e3-00120af9b1fd/volumes" Jul 01 23:10:45.632097 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.632067 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mq9fx" Jul 01 23:10:45.633747 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.633728 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6rb82" Jul 01 23:10:45.706635 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.706588 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" event={"ID":"0416fbc7-e700-4864-aa64-10b524e14ff9","Type":"ContainerStarted","Data":"3d12cbd740733e73c58df5d48d6e5f7d15c344f93ec6c59d1f428917ce7050a8"} Jul 01 23:10:45.706810 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.706692 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:10:45.711460 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.711428 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" event={"ID":"152fc74c-3a9c-40ec-ba5a-9d94c7848149","Type":"ContainerStarted","Data":"2cfa1416bcc9a0a164b18dbc318bd8629f6817604d225073c863a3d776dfc4ff"} Jul 01 23:10:45.711716 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.711689 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:10:45.726644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.726576 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" podStartSLOduration=2.330506206 podStartE2EDuration="5.726563164s" totalImagesPullingTime="3.396056958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:45.724450064 +0000 UTC m=+375.946978290" watchObservedRunningTime="2026-07-01 23:10:45.726563164 +0000 UTC m=+375.949091373" Jul 01 23:10:45.740977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:10:45.740937 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" podStartSLOduration=1.7626647210000002 podStartE2EDuration="5.740924506s" totalImagesPullingTime="3.978259785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:10:45.740190453 +0000 UTC m=+375.962718693" watchObservedRunningTime="2026-07-01 23:10:45.740924506 +0000 UTC m=+375.963452753" Jul 01 23:11:16.719180 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:16.719151 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rfhb5" Jul 01 23:11:16.722134 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:16.722116 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gn76v" Jul 01 23:11:29.030477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.030441 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ljkc6"] Jul 01 23:11:29.030943 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.030920 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" containerName="discovery" Jul 01 23:11:29.031126 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.031107 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" containerName="discovery" Jul 01 23:11:29.031283 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.031269 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c4aa64a-395a-436c-b3e3-00120af9b1fd" containerName="discovery" Jul 01 23:11:29.034429 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.034410 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:29.036888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.036868 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 01 23:11:29.036888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.036878 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-sqxcm\"" Jul 01 23:11:29.037599 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.037583 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 01 23:11:29.043011 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.042987 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ljkc6"] Jul 01 23:11:29.134080 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.134038 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42xh8\" (UniqueName: \"kubernetes.io/projected/42a9299a-f9a7-4e1c-a2ca-cbad037f537f-kube-api-access-42xh8\") pod \"kuadrant-operator-catalog-ljkc6\" (UID: \"42a9299a-f9a7-4e1c-a2ca-cbad037f537f\") " pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:29.234948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.234912 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42xh8\" (UniqueName: \"kubernetes.io/projected/42a9299a-f9a7-4e1c-a2ca-cbad037f537f-kube-api-access-42xh8\") pod \"kuadrant-operator-catalog-ljkc6\" (UID: \"42a9299a-f9a7-4e1c-a2ca-cbad037f537f\") " pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:29.243408 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.243381 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42xh8\" (UniqueName: \"kubernetes.io/projected/42a9299a-f9a7-4e1c-a2ca-cbad037f537f-kube-api-access-42xh8\") pod \"kuadrant-operator-catalog-ljkc6\" (UID: \"42a9299a-f9a7-4e1c-a2ca-cbad037f537f\") " pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:29.347546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.347516 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:29.398087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.398052 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ljkc6"] Jul 01 23:11:29.499679 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.499647 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ljkc6"] Jul 01 23:11:29.655175 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.655084 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4cmxt"] Jul 01 23:11:29.659822 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.659800 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:29.678831 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.678803 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4cmxt"] Jul 01 23:11:29.740367 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.740329 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcwgt\" (UniqueName: \"kubernetes.io/projected/81fd390d-b0de-4352-9de5-62d14e80b0a4-kube-api-access-qcwgt\") pod \"kuadrant-operator-catalog-4cmxt\" (UID: \"81fd390d-b0de-4352-9de5-62d14e80b0a4\") " pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:29.841848 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.841789 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcwgt\" (UniqueName: \"kubernetes.io/projected/81fd390d-b0de-4352-9de5-62d14e80b0a4-kube-api-access-qcwgt\") pod \"kuadrant-operator-catalog-4cmxt\" (UID: \"81fd390d-b0de-4352-9de5-62d14e80b0a4\") " pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:29.852501 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.852465 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcwgt\" (UniqueName: \"kubernetes.io/projected/81fd390d-b0de-4352-9de5-62d14e80b0a4-kube-api-access-qcwgt\") pod \"kuadrant-operator-catalog-4cmxt\" (UID: \"81fd390d-b0de-4352-9de5-62d14e80b0a4\") " pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:29.934355 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.934271 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" event={"ID":"42a9299a-f9a7-4e1c-a2ca-cbad037f537f","Type":"ContainerStarted","Data":"c50076506fc562aa32ca43bd37569ddbe6b7da1acbc2dd64bfc240d00aac69b1"} Jul 01 23:11:29.969696 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:29.969660 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:30.140956 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:30.140930 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4cmxt"] Jul 01 23:11:30.203968 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:11:30.203898 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81fd390d_b0de_4352_9de5_62d14e80b0a4.slice/crio-af8c3fe4f12675b5ae2a1efae8bd565584047c217a7611c3c246d9f0b0147503 WatchSource:0}: Error finding container af8c3fe4f12675b5ae2a1efae8bd565584047c217a7611c3c246d9f0b0147503: Status 404 returned error can't find the container with id af8c3fe4f12675b5ae2a1efae8bd565584047c217a7611c3c246d9f0b0147503 Jul 01 23:11:30.941415 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:30.941375 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" event={"ID":"81fd390d-b0de-4352-9de5-62d14e80b0a4","Type":"ContainerStarted","Data":"af8c3fe4f12675b5ae2a1efae8bd565584047c217a7611c3c246d9f0b0147503"} Jul 01 23:11:31.947796 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:31.947758 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" event={"ID":"42a9299a-f9a7-4e1c-a2ca-cbad037f537f","Type":"ContainerStarted","Data":"e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3"} Jul 01 23:11:31.948266 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:31.947768 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" podUID="42a9299a-f9a7-4e1c-a2ca-cbad037f537f" containerName="registry-server" containerID="cri-o://e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3" gracePeriod=2 Jul 01 23:11:31.949884 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:31.949861 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" event={"ID":"81fd390d-b0de-4352-9de5-62d14e80b0a4","Type":"ContainerStarted","Data":"a9ff5be647b615526959ada2f9c63ada22bfb49ff289a784b6271e9a15137c90"} Jul 01 23:11:31.966520 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:31.966470 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" podStartSLOduration=0.932048598 podStartE2EDuration="2.966451033s" totalImagesPullingTime="2.034402435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:11:31.965851032 +0000 UTC m=+422.188379268" watchObservedRunningTime="2026-07-01 23:11:31.966451033 +0000 UTC m=+422.188979245" Jul 01 23:11:31.982385 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:31.982339 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" podStartSLOduration=1.647831713 podStartE2EDuration="2.98232206s" totalImagesPullingTime="1.334490347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:11:31.981402909 +0000 UTC m=+422.203931119" watchObservedRunningTime="2026-07-01 23:11:31.98232206 +0000 UTC m=+422.204850271" Jul 01 23:11:32.190262 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.190235 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:32.271394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.271304 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42xh8\" (UniqueName: \"kubernetes.io/projected/42a9299a-f9a7-4e1c-a2ca-cbad037f537f-kube-api-access-42xh8\") pod \"42a9299a-f9a7-4e1c-a2ca-cbad037f537f\" (UID: \"42a9299a-f9a7-4e1c-a2ca-cbad037f537f\") " Jul 01 23:11:32.273454 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.273424 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a9299a-f9a7-4e1c-a2ca-cbad037f537f-kube-api-access-42xh8" (OuterVolumeSpecName: "kube-api-access-42xh8") pod "42a9299a-f9a7-4e1c-a2ca-cbad037f537f" (UID: "42a9299a-f9a7-4e1c-a2ca-cbad037f537f"). InnerVolumeSpecName "kube-api-access-42xh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:11:32.372012 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.371974 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42xh8\" (UniqueName: \"kubernetes.io/projected/42a9299a-f9a7-4e1c-a2ca-cbad037f537f-kube-api-access-42xh8\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:32.955970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.955933 2569 generic.go:356] "Generic (PLEG): container finished" podID="42a9299a-f9a7-4e1c-a2ca-cbad037f537f" containerID="e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3" exitCode=0 Jul 01 23:11:32.956362 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.955987 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" Jul 01 23:11:32.956362 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.956019 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" event={"ID":"42a9299a-f9a7-4e1c-a2ca-cbad037f537f","Type":"ContainerDied","Data":"e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3"} Jul 01 23:11:32.956362 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.956082 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ljkc6" event={"ID":"42a9299a-f9a7-4e1c-a2ca-cbad037f537f","Type":"ContainerDied","Data":"c50076506fc562aa32ca43bd37569ddbe6b7da1acbc2dd64bfc240d00aac69b1"} Jul 01 23:11:32.956362 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.956112 2569 scope.go:117] "RemoveContainer" containerID="e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3" Jul 01 23:11:32.967365 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.967345 2569 scope.go:117] "RemoveContainer" containerID="e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3" Jul 01 23:11:32.967648 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:11:32.967629 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3\": container with ID starting with e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3 not found: ID does not exist" containerID="e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3" Jul 01 23:11:32.967721 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.967660 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3"} err="failed to get container status \"e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3\": rpc error: code = NotFound desc = could not find container \"e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3\": container with ID starting with e4c44c762a0a3bfda7d1b2705fe17f23a7b7028f4cfbdc6c665dfb94365ba4b3 not found: ID does not exist" Jul 01 23:11:32.975128 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.975107 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ljkc6"] Jul 01 23:11:32.979083 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:32.979060 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ljkc6"] Jul 01 23:11:34.922574 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:34.922543 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a9299a-f9a7-4e1c-a2ca-cbad037f537f" path="/var/lib/kubelet/pods/42a9299a-f9a7-4e1c-a2ca-cbad037f537f/volumes" Jul 01 23:11:39.970038 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:39.970004 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:39.970445 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:39.970066 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:39.992478 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:39.992453 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:40.019523 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:40.019500 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4cmxt" Jul 01 23:11:47.208520 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.208485 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn"] Jul 01 23:11:47.209034 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.209017 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a9299a-f9a7-4e1c-a2ca-cbad037f537f" containerName="registry-server" Jul 01 23:11:47.209078 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.209035 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a9299a-f9a7-4e1c-a2ca-cbad037f537f" containerName="registry-server" Jul 01 23:11:47.209114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.209092 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a9299a-f9a7-4e1c-a2ca-cbad037f537f" containerName="registry-server" Jul 01 23:11:47.213693 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.213678 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.217516 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.217496 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kbgjj\"" Jul 01 23:11:47.228731 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.228709 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn"] Jul 01 23:11:47.309549 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.309525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.309718 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.309587 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.309718 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.309670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5xwv\" (UniqueName: \"kubernetes.io/projected/9df7591f-47f8-4f6e-a891-23dd5f0d156b-kube-api-access-s5xwv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.410922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.410881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.411080 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.410955 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5xwv\" (UniqueName: \"kubernetes.io/projected/9df7591f-47f8-4f6e-a891-23dd5f0d156b-kube-api-access-s5xwv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.411080 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.410992 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.411242 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.411220 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.411347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.411329 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.420290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.420266 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5xwv\" (UniqueName: \"kubernetes.io/projected/9df7591f-47f8-4f6e-a891-23dd5f0d156b-kube-api-access-s5xwv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.522447 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.522391 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:47.602266 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.602238 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn"] Jul 01 23:11:47.607643 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.607598 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.618551 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.618490 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn"] Jul 01 23:11:47.659990 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.659969 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn"] Jul 01 23:11:47.661947 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:11:47.661920 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df7591f_47f8_4f6e_a891_23dd5f0d156b.slice/crio-589677fe9b6e64993ec7eb0ed8c8e442c74f4bbc4f27f42d3a8d8a752af58cf4 WatchSource:0}: Error finding container 589677fe9b6e64993ec7eb0ed8c8e442c74f4bbc4f27f42d3a8d8a752af58cf4: Status 404 returned error can't find the container with id 589677fe9b6e64993ec7eb0ed8c8e442c74f4bbc4f27f42d3a8d8a752af58cf4 Jul 01 23:11:47.713632 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.713588 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.713737 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.713636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.713737 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.713654 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfng2\" (UniqueName: \"kubernetes.io/projected/644a2067-c25b-416a-83b3-a543f2842142-kube-api-access-vfng2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.814759 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.814736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.814867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.814770 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.814867 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.814792 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfng2\" (UniqueName: \"kubernetes.io/projected/644a2067-c25b-416a-83b3-a543f2842142-kube-api-access-vfng2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.815082 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.815063 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.815131 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.815104 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.823933 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.823910 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfng2\" (UniqueName: \"kubernetes.io/projected/644a2067-c25b-416a-83b3-a543f2842142-kube-api-access-vfng2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.921104 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.921077 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:47.988487 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.988457 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6"] Jul 01 23:11:47.993188 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:47.993169 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.002851 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.002808 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6"] Jul 01 23:11:48.037682 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.037643 2569 generic.go:356] "Generic (PLEG): container finished" podID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerID="4c48889d7f245b34cd5b9de4eb8ba7204b15cdc14dc1e58ca2b2b2dea82b38f0" exitCode=0 Jul 01 23:11:48.037836 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.037722 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" event={"ID":"9df7591f-47f8-4f6e-a891-23dd5f0d156b","Type":"ContainerDied","Data":"4c48889d7f245b34cd5b9de4eb8ba7204b15cdc14dc1e58ca2b2b2dea82b38f0"} Jul 01 23:11:48.037836 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.037780 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" event={"ID":"9df7591f-47f8-4f6e-a891-23dd5f0d156b","Type":"ContainerStarted","Data":"589677fe9b6e64993ec7eb0ed8c8e442c74f4bbc4f27f42d3a8d8a752af58cf4"} Jul 01 23:11:48.054511 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.054493 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn"] Jul 01 23:11:48.056713 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:11:48.056689 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644a2067_c25b_416a_83b3_a543f2842142.slice/crio-375fd2ce7972c7bd733b027526e2ed0526e5429cda89a0891d94d561d09b66cf WatchSource:0}: Error finding container 375fd2ce7972c7bd733b027526e2ed0526e5429cda89a0891d94d561d09b66cf: Status 404 returned error can't find the container with id 375fd2ce7972c7bd733b027526e2ed0526e5429cda89a0891d94d561d09b66cf Jul 01 23:11:48.118360 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.118337 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.118454 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.118375 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v87l6\" (UniqueName: \"kubernetes.io/projected/0255ff73-fd18-4f66-ba54-b96c83208dab-kube-api-access-v87l6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.118454 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.118401 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.219828 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.219786 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.220249 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.219845 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v87l6\" (UniqueName: \"kubernetes.io/projected/0255ff73-fd18-4f66-ba54-b96c83208dab-kube-api-access-v87l6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.220249 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.219884 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.220249 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.220137 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.220390 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.220283 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.230200 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.230179 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v87l6\" (UniqueName: \"kubernetes.io/projected/0255ff73-fd18-4f66-ba54-b96c83208dab-kube-api-access-v87l6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.304074 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.304047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:48.396159 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.396129 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk"] Jul 01 23:11:48.401576 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.401553 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.410038 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.410015 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk"] Jul 01 23:11:48.438445 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.438423 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6"] Jul 01 23:11:48.440366 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:11:48.440344 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0255ff73_fd18_4f66_ba54_b96c83208dab.slice/crio-11a58a9206cfb22e517576c2b873a70f79a003a4f781c4eafa05c303a133f402 WatchSource:0}: Error finding container 11a58a9206cfb22e517576c2b873a70f79a003a4f781c4eafa05c303a133f402: Status 404 returned error can't find the container with id 11a58a9206cfb22e517576c2b873a70f79a003a4f781c4eafa05c303a133f402 Jul 01 23:11:48.523578 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.523556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.523699 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.523623 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.523699 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.523650 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q97l5\" (UniqueName: \"kubernetes.io/projected/dad158f4-72d9-438f-9615-98445a2019e0-kube-api-access-q97l5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.624945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.624889 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.625031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.624954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.625031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.624982 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q97l5\" (UniqueName: \"kubernetes.io/projected/dad158f4-72d9-438f-9615-98445a2019e0-kube-api-access-q97l5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.625322 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.625303 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.625358 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.625329 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.634873 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.634851 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q97l5\" (UniqueName: \"kubernetes.io/projected/dad158f4-72d9-438f-9615-98445a2019e0-kube-api-access-q97l5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.716131 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.716108 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:48.849779 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:48.849735 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk"] Jul 01 23:11:48.972795 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:11:48.972716 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad158f4_72d9_438f_9615_98445a2019e0.slice/crio-8c6babb3aa2443057f389d52dd3bbaf9e05c00f1e4d3157e74270848c5b5eca7 WatchSource:0}: Error finding container 8c6babb3aa2443057f389d52dd3bbaf9e05c00f1e4d3157e74270848c5b5eca7: Status 404 returned error can't find the container with id 8c6babb3aa2443057f389d52dd3bbaf9e05c00f1e4d3157e74270848c5b5eca7 Jul 01 23:11:49.046088 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.046058 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" event={"ID":"dad158f4-72d9-438f-9615-98445a2019e0","Type":"ContainerStarted","Data":"2878505e9dfe1d01e7e7ae3b5c53eb1728db5930ce8d07b7988704742f056d79"} Jul 01 23:11:49.046177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.046120 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" event={"ID":"dad158f4-72d9-438f-9615-98445a2019e0","Type":"ContainerStarted","Data":"8c6babb3aa2443057f389d52dd3bbaf9e05c00f1e4d3157e74270848c5b5eca7"} Jul 01 23:11:49.047565 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.047542 2569 generic.go:356] "Generic (PLEG): container finished" podID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerID="804f7e3132135347c9210dd4605c70842922fc02ef91443397547ac06e73b871" exitCode=0 Jul 01 23:11:49.047670 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.047631 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" event={"ID":"0255ff73-fd18-4f66-ba54-b96c83208dab","Type":"ContainerDied","Data":"804f7e3132135347c9210dd4605c70842922fc02ef91443397547ac06e73b871"} Jul 01 23:11:49.047670 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.047665 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" event={"ID":"0255ff73-fd18-4f66-ba54-b96c83208dab","Type":"ContainerStarted","Data":"11a58a9206cfb22e517576c2b873a70f79a003a4f781c4eafa05c303a133f402"} Jul 01 23:11:49.049306 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.049281 2569 generic.go:356] "Generic (PLEG): container finished" podID="644a2067-c25b-416a-83b3-a543f2842142" containerID="510e1866ee5c261faed4689ace45646b43879b7683bde8db1f76cea82ce38eb5" exitCode=0 Jul 01 23:11:49.049412 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.049395 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" event={"ID":"644a2067-c25b-416a-83b3-a543f2842142","Type":"ContainerDied","Data":"510e1866ee5c261faed4689ace45646b43879b7683bde8db1f76cea82ce38eb5"} Jul 01 23:11:49.049481 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:49.049439 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" event={"ID":"644a2067-c25b-416a-83b3-a543f2842142","Type":"ContainerStarted","Data":"375fd2ce7972c7bd733b027526e2ed0526e5429cda89a0891d94d561d09b66cf"} Jul 01 23:11:50.057450 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:50.057368 2569 generic.go:356] "Generic (PLEG): container finished" podID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerID="21c87dece706ce343b8e0af0f58dcfda1d86c6f89de8303c6bbb20a79113cf11" exitCode=0 Jul 01 23:11:50.057848 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:50.057448 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" event={"ID":"9df7591f-47f8-4f6e-a891-23dd5f0d156b","Type":"ContainerDied","Data":"21c87dece706ce343b8e0af0f58dcfda1d86c6f89de8303c6bbb20a79113cf11"} Jul 01 23:11:50.058957 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:50.058915 2569 generic.go:356] "Generic (PLEG): container finished" podID="dad158f4-72d9-438f-9615-98445a2019e0" containerID="2878505e9dfe1d01e7e7ae3b5c53eb1728db5930ce8d07b7988704742f056d79" exitCode=0 Jul 01 23:11:50.059018 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:50.058992 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" event={"ID":"dad158f4-72d9-438f-9615-98445a2019e0","Type":"ContainerDied","Data":"2878505e9dfe1d01e7e7ae3b5c53eb1728db5930ce8d07b7988704742f056d79"} Jul 01 23:11:50.060842 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:50.060818 2569 generic.go:356] "Generic (PLEG): container finished" podID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerID="986c30eda41c1128cbfd31422f477ee3051105ee7eb590ce3880a6ae2b96400a" exitCode=0 Jul 01 23:11:50.060934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:50.060906 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" event={"ID":"0255ff73-fd18-4f66-ba54-b96c83208dab","Type":"ContainerDied","Data":"986c30eda41c1128cbfd31422f477ee3051105ee7eb590ce3880a6ae2b96400a"} Jul 01 23:11:51.067268 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.067237 2569 generic.go:356] "Generic (PLEG): container finished" podID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerID="f1ed34025bdceeaeaf6b657c91a0e53f91672e2d1f5554c78a9c56ef1e94a327" exitCode=0 Jul 01 23:11:51.067596 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.067317 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" event={"ID":"9df7591f-47f8-4f6e-a891-23dd5f0d156b","Type":"ContainerDied","Data":"f1ed34025bdceeaeaf6b657c91a0e53f91672e2d1f5554c78a9c56ef1e94a327"} Jul 01 23:11:51.068961 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.068942 2569 generic.go:356] "Generic (PLEG): container finished" podID="dad158f4-72d9-438f-9615-98445a2019e0" containerID="463b0ea50bcbd926d32d3273d3db05d61883df3d9b1213a596d4340c29b6b608" exitCode=0 Jul 01 23:11:51.069040 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.069003 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" event={"ID":"dad158f4-72d9-438f-9615-98445a2019e0","Type":"ContainerDied","Data":"463b0ea50bcbd926d32d3273d3db05d61883df3d9b1213a596d4340c29b6b608"} Jul 01 23:11:51.071137 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.071116 2569 generic.go:356] "Generic (PLEG): container finished" podID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerID="a3eefcc4bb3371214fc6b469b30b3ff28da777411cb394d3cb115f4943b56d62" exitCode=0 Jul 01 23:11:51.071220 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.071189 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" event={"ID":"0255ff73-fd18-4f66-ba54-b96c83208dab","Type":"ContainerDied","Data":"a3eefcc4bb3371214fc6b469b30b3ff28da777411cb394d3cb115f4943b56d62"} Jul 01 23:11:51.073065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.073041 2569 generic.go:356] "Generic (PLEG): container finished" podID="644a2067-c25b-416a-83b3-a543f2842142" containerID="a9b6a9d48718e7392cc76fe828239fcdd5e45515003c07d3e457e98d0c04f416" exitCode=0 Jul 01 23:11:51.073151 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:51.073095 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" event={"ID":"644a2067-c25b-416a-83b3-a543f2842142","Type":"ContainerDied","Data":"a9b6a9d48718e7392cc76fe828239fcdd5e45515003c07d3e457e98d0c04f416"} Jul 01 23:11:52.080764 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.080689 2569 generic.go:356] "Generic (PLEG): container finished" podID="dad158f4-72d9-438f-9615-98445a2019e0" containerID="37bc00dd651e52e608480ed1fbf0e4e2b672d92c93a0b601abc8f2265c188d73" exitCode=0 Jul 01 23:11:52.081151 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.080784 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" event={"ID":"dad158f4-72d9-438f-9615-98445a2019e0","Type":"ContainerDied","Data":"37bc00dd651e52e608480ed1fbf0e4e2b672d92c93a0b601abc8f2265c188d73"} Jul 01 23:11:52.082833 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.082810 2569 generic.go:356] "Generic (PLEG): container finished" podID="644a2067-c25b-416a-83b3-a543f2842142" containerID="f5e191d3d1b857be0f2069c7a68e68621b5ef18c7913b3c1dbee64ee6da93206" exitCode=0 Jul 01 23:11:52.082956 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.082890 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" event={"ID":"644a2067-c25b-416a-83b3-a543f2842142","Type":"ContainerDied","Data":"f5e191d3d1b857be0f2069c7a68e68621b5ef18c7913b3c1dbee64ee6da93206"} Jul 01 23:11:52.216693 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.216666 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:52.249337 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.249313 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:52.359934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.359864 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-bundle\") pod \"0255ff73-fd18-4f66-ba54-b96c83208dab\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " Jul 01 23:11:52.359934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.359929 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-util\") pod \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " Jul 01 23:11:52.360152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.359956 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5xwv\" (UniqueName: \"kubernetes.io/projected/9df7591f-47f8-4f6e-a891-23dd5f0d156b-kube-api-access-s5xwv\") pod \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " Jul 01 23:11:52.360152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.360012 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v87l6\" (UniqueName: \"kubernetes.io/projected/0255ff73-fd18-4f66-ba54-b96c83208dab-kube-api-access-v87l6\") pod \"0255ff73-fd18-4f66-ba54-b96c83208dab\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " Jul 01 23:11:52.360152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.360033 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-util\") pod \"0255ff73-fd18-4f66-ba54-b96c83208dab\" (UID: \"0255ff73-fd18-4f66-ba54-b96c83208dab\") " Jul 01 23:11:52.360152 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.360049 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-bundle\") pod \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\" (UID: \"9df7591f-47f8-4f6e-a891-23dd5f0d156b\") " Jul 01 23:11:52.360522 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.360490 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-bundle" (OuterVolumeSpecName: "bundle") pod "0255ff73-fd18-4f66-ba54-b96c83208dab" (UID: "0255ff73-fd18-4f66-ba54-b96c83208dab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:52.360661 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.360556 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-bundle" (OuterVolumeSpecName: "bundle") pod "9df7591f-47f8-4f6e-a891-23dd5f0d156b" (UID: "9df7591f-47f8-4f6e-a891-23dd5f0d156b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:52.362223 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.362196 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df7591f-47f8-4f6e-a891-23dd5f0d156b-kube-api-access-s5xwv" (OuterVolumeSpecName: "kube-api-access-s5xwv") pod "9df7591f-47f8-4f6e-a891-23dd5f0d156b" (UID: "9df7591f-47f8-4f6e-a891-23dd5f0d156b"). InnerVolumeSpecName "kube-api-access-s5xwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:11:52.362512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.362491 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0255ff73-fd18-4f66-ba54-b96c83208dab-kube-api-access-v87l6" (OuterVolumeSpecName: "kube-api-access-v87l6") pod "0255ff73-fd18-4f66-ba54-b96c83208dab" (UID: "0255ff73-fd18-4f66-ba54-b96c83208dab"). InnerVolumeSpecName "kube-api-access-v87l6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:11:52.365600 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.365571 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-util" (OuterVolumeSpecName: "util") pod "9df7591f-47f8-4f6e-a891-23dd5f0d156b" (UID: "9df7591f-47f8-4f6e-a891-23dd5f0d156b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:52.366488 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.366468 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-util" (OuterVolumeSpecName: "util") pod "0255ff73-fd18-4f66-ba54-b96c83208dab" (UID: "0255ff73-fd18-4f66-ba54-b96c83208dab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:52.460581 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.460555 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v87l6\" (UniqueName: \"kubernetes.io/projected/0255ff73-fd18-4f66-ba54-b96c83208dab-kube-api-access-v87l6\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:52.460581 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.460578 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:52.460727 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.460587 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:52.460727 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.460595 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0255ff73-fd18-4f66-ba54-b96c83208dab-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:52.460727 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.460602 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9df7591f-47f8-4f6e-a891-23dd5f0d156b-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:52.460727 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:52.460627 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5xwv\" (UniqueName: \"kubernetes.io/projected/9df7591f-47f8-4f6e-a891-23dd5f0d156b-kube-api-access-s5xwv\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:53.090107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.090079 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" Jul 01 23:11:53.090107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.090099 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6" event={"ID":"0255ff73-fd18-4f66-ba54-b96c83208dab","Type":"ContainerDied","Data":"11a58a9206cfb22e517576c2b873a70f79a003a4f781c4eafa05c303a133f402"} Jul 01 23:11:53.090537 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.090141 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11a58a9206cfb22e517576c2b873a70f79a003a4f781c4eafa05c303a133f402" Jul 01 23:11:53.092367 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.092348 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" Jul 01 23:11:53.092485 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.092366 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn" event={"ID":"9df7591f-47f8-4f6e-a891-23dd5f0d156b","Type":"ContainerDied","Data":"589677fe9b6e64993ec7eb0ed8c8e442c74f4bbc4f27f42d3a8d8a752af58cf4"} Jul 01 23:11:53.092485 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.092417 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="589677fe9b6e64993ec7eb0ed8c8e442c74f4bbc4f27f42d3a8d8a752af58cf4" Jul 01 23:11:53.256143 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.256122 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:53.259359 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.259336 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:53.368863 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.368833 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-util\") pod \"644a2067-c25b-416a-83b3-a543f2842142\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " Jul 01 23:11:53.369014 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.368906 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfng2\" (UniqueName: \"kubernetes.io/projected/644a2067-c25b-416a-83b3-a543f2842142-kube-api-access-vfng2\") pod \"644a2067-c25b-416a-83b3-a543f2842142\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " Jul 01 23:11:53.369014 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.368924 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-util\") pod \"dad158f4-72d9-438f-9615-98445a2019e0\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " Jul 01 23:11:53.369014 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.368948 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q97l5\" (UniqueName: \"kubernetes.io/projected/dad158f4-72d9-438f-9615-98445a2019e0-kube-api-access-q97l5\") pod \"dad158f4-72d9-438f-9615-98445a2019e0\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " Jul 01 23:11:53.369014 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.368967 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-bundle\") pod \"644a2067-c25b-416a-83b3-a543f2842142\" (UID: \"644a2067-c25b-416a-83b3-a543f2842142\") " Jul 01 23:11:53.369014 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.368988 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-bundle\") pod \"dad158f4-72d9-438f-9615-98445a2019e0\" (UID: \"dad158f4-72d9-438f-9615-98445a2019e0\") " Jul 01 23:11:53.369585 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.369560 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-bundle" (OuterVolumeSpecName: "bundle") pod "644a2067-c25b-416a-83b3-a543f2842142" (UID: "644a2067-c25b-416a-83b3-a543f2842142"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:53.369873 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.369843 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-bundle" (OuterVolumeSpecName: "bundle") pod "dad158f4-72d9-438f-9615-98445a2019e0" (UID: "dad158f4-72d9-438f-9615-98445a2019e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:53.371182 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.371160 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/644a2067-c25b-416a-83b3-a543f2842142-kube-api-access-vfng2" (OuterVolumeSpecName: "kube-api-access-vfng2") pod "644a2067-c25b-416a-83b3-a543f2842142" (UID: "644a2067-c25b-416a-83b3-a543f2842142"). InnerVolumeSpecName "kube-api-access-vfng2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:11:53.371403 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.371377 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dad158f4-72d9-438f-9615-98445a2019e0-kube-api-access-q97l5" (OuterVolumeSpecName: "kube-api-access-q97l5") pod "dad158f4-72d9-438f-9615-98445a2019e0" (UID: "dad158f4-72d9-438f-9615-98445a2019e0"). InnerVolumeSpecName "kube-api-access-q97l5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:11:53.374568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.374548 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-util" (OuterVolumeSpecName: "util") pod "644a2067-c25b-416a-83b3-a543f2842142" (UID: "644a2067-c25b-416a-83b3-a543f2842142"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:53.375062 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.375043 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-util" (OuterVolumeSpecName: "util") pod "dad158f4-72d9-438f-9615-98445a2019e0" (UID: "dad158f4-72d9-438f-9615-98445a2019e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:11:53.470101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.470043 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:53.470101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.470065 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfng2\" (UniqueName: \"kubernetes.io/projected/644a2067-c25b-416a-83b3-a543f2842142-kube-api-access-vfng2\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:53.470101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.470075 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:53.470101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.470084 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q97l5\" (UniqueName: \"kubernetes.io/projected/dad158f4-72d9-438f-9615-98445a2019e0-kube-api-access-q97l5\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:53.470101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.470092 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/644a2067-c25b-416a-83b3-a543f2842142-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:53.470101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:53.470099 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dad158f4-72d9-438f-9615-98445a2019e0-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:11:54.100491 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:54.100466 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" Jul 01 23:11:54.100944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:54.100470 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk" event={"ID":"dad158f4-72d9-438f-9615-98445a2019e0","Type":"ContainerDied","Data":"8c6babb3aa2443057f389d52dd3bbaf9e05c00f1e4d3157e74270848c5b5eca7"} Jul 01 23:11:54.100944 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:54.100574 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c6babb3aa2443057f389d52dd3bbaf9e05c00f1e4d3157e74270848c5b5eca7" Jul 01 23:11:54.102383 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:54.102366 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" Jul 01 23:11:54.102477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:54.102396 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn" event={"ID":"644a2067-c25b-416a-83b3-a543f2842142","Type":"ContainerDied","Data":"375fd2ce7972c7bd733b027526e2ed0526e5429cda89a0891d94d561d09b66cf"} Jul 01 23:11:54.102477 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:11:54.102427 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="375fd2ce7972c7bd733b027526e2ed0526e5429cda89a0891d94d561d09b66cf" Jul 01 23:12:03.317864 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.317825 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5"] Jul 01 23:12:03.318218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318196 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="pull" Jul 01 23:12:03.318218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318207 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="pull" Jul 01 23:12:03.318218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318216 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="extract" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318222 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="extract" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318236 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318242 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318251 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="extract" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318257 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="extract" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318263 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318268 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318277 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="pull" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318282 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="pull" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318288 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="pull" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318293 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="pull" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318298 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="extract" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318304 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="extract" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318310 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318315 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318321 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318328 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="util" Jul 01 23:12:03.318326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318334 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="extract" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318339 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="extract" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318350 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="pull" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318355 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="pull" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318405 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df7591f-47f8-4f6e-a891-23dd5f0d156b" containerName="extract" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318412 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dad158f4-72d9-438f-9615-98445a2019e0" containerName="extract" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318421 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0255ff73-fd18-4f66-ba54-b96c83208dab" containerName="extract" Jul 01 23:12:03.318849 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.318427 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="644a2067-c25b-416a-83b3-a543f2842142" containerName="extract" Jul 01 23:12:03.323736 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.323711 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:03.326465 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.326445 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-24lzq\"" Jul 01 23:12:03.332547 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.332523 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5"] Jul 01 23:12:03.454009 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.453963 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fk5c\" (UniqueName: \"kubernetes.io/projected/27c4236a-5a70-4009-9617-b8f7dbb7d915-kube-api-access-7fk5c\") pod \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" (UID: \"27c4236a-5a70-4009-9617-b8f7dbb7d915\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:03.555387 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.555353 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fk5c\" (UniqueName: \"kubernetes.io/projected/27c4236a-5a70-4009-9617-b8f7dbb7d915-kube-api-access-7fk5c\") pod \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" (UID: \"27c4236a-5a70-4009-9617-b8f7dbb7d915\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:03.567454 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.567421 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fk5c\" (UniqueName: \"kubernetes.io/projected/27c4236a-5a70-4009-9617-b8f7dbb7d915-kube-api-access-7fk5c\") pod \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" (UID: \"27c4236a-5a70-4009-9617-b8f7dbb7d915\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:03.635769 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.635685 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:03.778272 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:03.778192 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5"] Jul 01 23:12:03.780677 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:03.780650 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c4236a_5a70_4009_9617_b8f7dbb7d915.slice/crio-a8affbf0aafd7ba648ab2f58a6f09d4a82c218ea64d7f8af13fb4470583bb7e0 WatchSource:0}: Error finding container a8affbf0aafd7ba648ab2f58a6f09d4a82c218ea64d7f8af13fb4470583bb7e0: Status 404 returned error can't find the container with id a8affbf0aafd7ba648ab2f58a6f09d4a82c218ea64d7f8af13fb4470583bb7e0 Jul 01 23:12:04.157479 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:04.157446 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" event={"ID":"27c4236a-5a70-4009-9617-b8f7dbb7d915","Type":"ContainerStarted","Data":"a8affbf0aafd7ba648ab2f58a6f09d4a82c218ea64d7f8af13fb4470583bb7e0"} Jul 01 23:12:06.169482 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:06.169440 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" event={"ID":"27c4236a-5a70-4009-9617-b8f7dbb7d915","Type":"ContainerStarted","Data":"446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61"} Jul 01 23:12:06.169899 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:06.169627 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:06.190297 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:06.190249 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" podStartSLOduration=1.39177857 podStartE2EDuration="3.190233668s" totalImagesPullingTime="1.798455098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:06.189053811 +0000 UTC m=+456.411582053" watchObservedRunningTime="2026-07-01 23:12:06.190233668 +0000 UTC m=+456.412761882" Jul 01 23:12:08.262215 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.262184 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp"] Jul 01 23:12:08.266494 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.266479 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.268843 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.268828 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sszww\"" Jul 01 23:12:08.277884 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.277861 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp"] Jul 01 23:12:08.296336 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.296309 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6t4\" (UniqueName: \"kubernetes.io/projected/e07cd1c0-8cc2-49b6-b97d-7a733a274331-kube-api-access-gt6t4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.296432 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.296358 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e07cd1c0-8cc2-49b6-b97d-7a733a274331-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.397177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.397150 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6t4\" (UniqueName: \"kubernetes.io/projected/e07cd1c0-8cc2-49b6-b97d-7a733a274331-kube-api-access-gt6t4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.397303 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.397197 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e07cd1c0-8cc2-49b6-b97d-7a733a274331-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.397639 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.397593 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e07cd1c0-8cc2-49b6-b97d-7a733a274331-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.406276 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.406248 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6t4\" (UniqueName: \"kubernetes.io/projected/e07cd1c0-8cc2-49b6-b97d-7a733a274331-kube-api-access-gt6t4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.577763 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.577741 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:08.715068 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:08.715045 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp"] Jul 01 23:12:08.717148 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:08.717127 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07cd1c0_8cc2_49b6_b97d_7a733a274331.slice/crio-836284e5979cf2e350f7d73cd2e9fd87d0c1fc73ce0becbfa6f68c9323f5ed76 WatchSource:0}: Error finding container 836284e5979cf2e350f7d73cd2e9fd87d0c1fc73ce0becbfa6f68c9323f5ed76: Status 404 returned error can't find the container with id 836284e5979cf2e350f7d73cd2e9fd87d0c1fc73ce0becbfa6f68c9323f5ed76 Jul 01 23:12:09.187234 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:09.187201 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" event={"ID":"e07cd1c0-8cc2-49b6-b97d-7a733a274331","Type":"ContainerStarted","Data":"836284e5979cf2e350f7d73cd2e9fd87d0c1fc73ce0becbfa6f68c9323f5ed76"} Jul 01 23:12:10.819442 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:10.819412 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf"] Jul 01 23:12:10.823051 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:10.823036 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:10.848815 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:10.848787 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf"] Jul 01 23:12:10.849924 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:10.849902 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-djlxf\"" Jul 01 23:12:10.917524 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:10.917495 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cz4r\" (UniqueName: \"kubernetes.io/projected/7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9-kube-api-access-5cz4r\") pod \"authorino-operator-75cc8d5cb7-jdfvf\" (UID: \"7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:11.018662 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:11.018598 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cz4r\" (UniqueName: \"kubernetes.io/projected/7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9-kube-api-access-5cz4r\") pod \"authorino-operator-75cc8d5cb7-jdfvf\" (UID: \"7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:11.031156 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:11.031119 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cz4r\" (UniqueName: \"kubernetes.io/projected/7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9-kube-api-access-5cz4r\") pod \"authorino-operator-75cc8d5cb7-jdfvf\" (UID: \"7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:11.133746 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:11.133659 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:11.273872 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:11.273844 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf"] Jul 01 23:12:11.275875 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:11.275839 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa7f8e1_9aac_48a6_82d4_361d4ed50fc9.slice/crio-abc4a72efa19439cf4a847cfd9b9fbc0cd09292fe48d395d63757802366e0ab7 WatchSource:0}: Error finding container abc4a72efa19439cf4a847cfd9b9fbc0cd09292fe48d395d63757802366e0ab7: Status 404 returned error can't find the container with id abc4a72efa19439cf4a847cfd9b9fbc0cd09292fe48d395d63757802366e0ab7 Jul 01 23:12:12.208473 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:12.208435 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" event={"ID":"7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9","Type":"ContainerStarted","Data":"abc4a72efa19439cf4a847cfd9b9fbc0cd09292fe48d395d63757802366e0ab7"} Jul 01 23:12:14.051235 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.049364 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht"] Jul 01 23:12:14.058390 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.058367 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:14.061177 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.061155 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vfqlj\"" Jul 01 23:12:14.061338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.061314 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 01 23:12:14.066449 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.066427 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht"] Jul 01 23:12:14.153330 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.153298 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdckj\" (UniqueName: \"kubernetes.io/projected/98a2becf-f5ff-46d8-8d67-a3030a5ae191-kube-api-access-cdckj\") pod \"dns-operator-controller-manager-864dd78c4c-qb9ht\" (UID: \"98a2becf-f5ff-46d8-8d67-a3030a5ae191\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:14.225684 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.225646 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" event={"ID":"7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9","Type":"ContainerStarted","Data":"426ad6ae000f05b9481b061eeb021b3227d039540ea036ed5dfb6786ca2578bc"} Jul 01 23:12:14.225873 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.225854 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:14.247081 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.247036 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" podStartSLOduration=1.854916727 podStartE2EDuration="4.247021798s" totalImagesPullingTime="2.392105071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:14.245337757 +0000 UTC m=+464.467865967" watchObservedRunningTime="2026-07-01 23:12:14.247021798 +0000 UTC m=+464.469550025" Jul 01 23:12:14.254761 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.254735 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdckj\" (UniqueName: \"kubernetes.io/projected/98a2becf-f5ff-46d8-8d67-a3030a5ae191-kube-api-access-cdckj\") pod \"dns-operator-controller-manager-864dd78c4c-qb9ht\" (UID: \"98a2becf-f5ff-46d8-8d67-a3030a5ae191\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:14.266488 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.266459 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdckj\" (UniqueName: \"kubernetes.io/projected/98a2becf-f5ff-46d8-8d67-a3030a5ae191-kube-api-access-cdckj\") pod \"dns-operator-controller-manager-864dd78c4c-qb9ht\" (UID: \"98a2becf-f5ff-46d8-8d67-a3030a5ae191\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:14.372532 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:14.372504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:15.098707 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:15.098673 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a2becf_f5ff_46d8_8d67_a3030a5ae191.slice/crio-d9b5286e2973e124e79886d4d341401e8d92a5f480659fe8ddc9d583863d0f8d WatchSource:0}: Error finding container d9b5286e2973e124e79886d4d341401e8d92a5f480659fe8ddc9d583863d0f8d: Status 404 returned error can't find the container with id d9b5286e2973e124e79886d4d341401e8d92a5f480659fe8ddc9d583863d0f8d Jul 01 23:12:15.099065 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.098883 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht"] Jul 01 23:12:15.233128 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.233041 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" event={"ID":"e07cd1c0-8cc2-49b6-b97d-7a733a274331","Type":"ContainerStarted","Data":"34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0"} Jul 01 23:12:15.233338 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.233141 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:15.234326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.234303 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" event={"ID":"98a2becf-f5ff-46d8-8d67-a3030a5ae191","Type":"ContainerStarted","Data":"d9b5286e2973e124e79886d4d341401e8d92a5f480659fe8ddc9d583863d0f8d"} Jul 01 23:12:15.261084 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.261038 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" podStartSLOduration=0.961169555 podStartE2EDuration="7.261025853s" totalImagesPullingTime="6.299856298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:15.259134937 +0000 UTC m=+465.481663150" watchObservedRunningTime="2026-07-01 23:12:15.261025853 +0000 UTC m=+465.483554063" Jul 01 23:12:15.449560 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.449528 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77dd554656-mbmf2"] Jul 01 23:12:15.453310 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.453291 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.475512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.475477 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dd554656-mbmf2"] Jul 01 23:12:15.566659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566623 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/019f2a75-0405-4f2d-8128-d65b541ef709-console-serving-cert\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.566659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566660 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-service-ca\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.566906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp7ws\" (UniqueName: \"kubernetes.io/projected/019f2a75-0405-4f2d-8128-d65b541ef709-kube-api-access-jp7ws\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.566906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566755 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/019f2a75-0405-4f2d-8128-d65b541ef709-console-oauth-config\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.566906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566793 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-console-config\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.566906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566837 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-oauth-serving-cert\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.566906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.566876 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-trusted-ca-bundle\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.667977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.667869 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/019f2a75-0405-4f2d-8128-d65b541ef709-console-serving-cert\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.667977 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.667917 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-service-ca\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.668290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668057 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp7ws\" (UniqueName: \"kubernetes.io/projected/019f2a75-0405-4f2d-8128-d65b541ef709-kube-api-access-jp7ws\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.668290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668108 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/019f2a75-0405-4f2d-8128-d65b541ef709-console-oauth-config\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.668290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668135 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-console-config\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.668290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668174 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-oauth-serving-cert\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.668290 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668221 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-trusted-ca-bundle\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.668880 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668730 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-service-ca\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.669344 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.668976 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-console-config\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.669404 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.669393 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-trusted-ca-bundle\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.669493 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.669436 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/019f2a75-0405-4f2d-8128-d65b541ef709-oauth-serving-cert\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.671031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.670996 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/019f2a75-0405-4f2d-8128-d65b541ef709-console-oauth-config\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.671119 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.671103 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/019f2a75-0405-4f2d-8128-d65b541ef709-console-serving-cert\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.680700 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.680676 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp7ws\" (UniqueName: \"kubernetes.io/projected/019f2a75-0405-4f2d-8128-d65b541ef709-kube-api-access-jp7ws\") pod \"console-77dd554656-mbmf2\" (UID: \"019f2a75-0405-4f2d-8128-d65b541ef709\") " pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.763417 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.763379 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:15.902215 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:15.902185 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dd554656-mbmf2"] Jul 01 23:12:15.903538 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:15.903516 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019f2a75_0405_4f2d_8128_d65b541ef709.slice/crio-20295da08546ae6c3d377a9fb43160cd0aba9361186413f41283fea07acc9f56 WatchSource:0}: Error finding container 20295da08546ae6c3d377a9fb43160cd0aba9361186413f41283fea07acc9f56: Status 404 returned error can't find the container with id 20295da08546ae6c3d377a9fb43160cd0aba9361186413f41283fea07acc9f56 Jul 01 23:12:16.242546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:16.242447 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dd554656-mbmf2" event={"ID":"019f2a75-0405-4f2d-8128-d65b541ef709","Type":"ContainerStarted","Data":"5c231ed223393a721b1f69a278df8af3a2bc29d4345953ff03b7fa7e145d8808"} Jul 01 23:12:16.242546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:16.242507 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dd554656-mbmf2" event={"ID":"019f2a75-0405-4f2d-8128-d65b541ef709","Type":"ContainerStarted","Data":"20295da08546ae6c3d377a9fb43160cd0aba9361186413f41283fea07acc9f56"} Jul 01 23:12:16.263388 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:16.263338 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77dd554656-mbmf2" podStartSLOduration=1.263317762 podStartE2EDuration="1.263317762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:16.261035229 +0000 UTC m=+466.483563461" watchObservedRunningTime="2026-07-01 23:12:16.263317762 +0000 UTC m=+466.485845953" Jul 01 23:12:17.177470 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:17.177437 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:18.256754 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:18.256715 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" event={"ID":"98a2becf-f5ff-46d8-8d67-a3030a5ae191","Type":"ContainerStarted","Data":"47bdeb0063e1d24af0ce9c485c1bf4310c929f2e6eda0561373d81f5abc7fd70"} Jul 01 23:12:18.257169 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:18.256844 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:18.282435 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:18.282372 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" podStartSLOduration=2.060990688 podStartE2EDuration="4.282353076s" totalImagesPullingTime="2.221362388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:18.277198135 +0000 UTC m=+468.499726370" watchObservedRunningTime="2026-07-01 23:12:18.282353076 +0000 UTC m=+468.504881294" Jul 01 23:12:25.236597 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:25.236560 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jdfvf" Jul 01 23:12:25.764360 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:25.764326 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:25.764541 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:25.764383 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:25.768961 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:25.768935 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:26.245060 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:26.245035 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:26.303701 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:26.303674 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77dd554656-mbmf2" Jul 01 23:12:26.370359 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:26.370330 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d65b8d58d-slbgt"] Jul 01 23:12:27.365738 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.365706 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76"] Jul 01 23:12:27.369480 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.369444 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.382766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.382745 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76"] Jul 01 23:12:27.468589 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.468554 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76"] Jul 01 23:12:27.469463 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:27.469421 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-8x8bj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" Jul 01 23:12:27.470394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.470367 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x8bj\" (UniqueName: \"kubernetes.io/projected/3ceb73e4-19fc-4cba-b68c-55616bf20658-kube-api-access-8x8bj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.470519 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.470415 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ceb73e4-19fc-4cba-b68c-55616bf20658-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.571105 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.571068 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x8bj\" (UniqueName: \"kubernetes.io/projected/3ceb73e4-19fc-4cba-b68c-55616bf20658-kube-api-access-8x8bj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.571105 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.571111 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ceb73e4-19fc-4cba-b68c-55616bf20658-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.571487 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.571467 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ceb73e4-19fc-4cba-b68c-55616bf20658-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.580504 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.580482 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x8bj\" (UniqueName: \"kubernetes.io/projected/3ceb73e4-19fc-4cba-b68c-55616bf20658-kube-api-access-8x8bj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:27.938775 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.938736 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76"] Jul 01 23:12:27.945326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.945302 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76"] Jul 01 23:12:27.959705 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.959679 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp"] Jul 01 23:12:27.959886 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.959863 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" containerName="manager" containerID="cri-o://34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0" gracePeriod=2 Jul 01 23:12:27.971032 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.971012 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp"] Jul 01 23:12:27.989841 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.989815 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5"] Jul 01 23:12:27.990076 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.990046 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" containerName="manager" containerID="cri-o://446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61" gracePeriod=2 Jul 01 23:12:27.992538 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:27.992491 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:27.996398 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:27.994358 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:27.996398 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.994401 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw"] Jul 01 23:12:27.996398 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.994895 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" containerName="manager" Jul 01 23:12:27.996398 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.994912 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" containerName="manager" Jul 01 23:12:27.996398 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.995045 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" containerName="manager" Jul 01 23:12:27.998457 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.998440 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5"] Jul 01 23:12:27.998542 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:27.998521 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.000310 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.000273 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.017686 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.017658 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw"] Jul 01 23:12:28.037858 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.037829 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q"] Jul 01 23:12:28.038455 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.038439 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" containerName="manager" Jul 01 23:12:28.038510 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.038458 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" containerName="manager" Jul 01 23:12:28.038587 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.038577 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" containerName="manager" Jul 01 23:12:28.041642 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.041599 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.042304 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.042290 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:28.044083 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.044054 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.045853 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.045831 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.054238 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.054218 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q"] Jul 01 23:12:28.076329 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.076307 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78248e42-57ee-4ed2-bbe4-928af2067507-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbqhw\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.076450 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.076348 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxq56\" (UniqueName: \"kubernetes.io/projected/78248e42-57ee-4ed2-bbe4-928af2067507-kube-api-access-zxq56\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbqhw\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.177548 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.177513 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxq56\" (UniqueName: \"kubernetes.io/projected/78248e42-57ee-4ed2-bbe4-928af2067507-kube-api-access-zxq56\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbqhw\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.177713 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.177641 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78248e42-57ee-4ed2-bbe4-928af2067507-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbqhw\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.177713 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.177672 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5tfl\" (UniqueName: \"kubernetes.io/projected/3fe29758-690b-4ea7-83af-b2c70b9db594-kube-api-access-k5tfl\") pod \"limitador-operator-controller-manager-859d7b78bd-f4b8q\" (UID: \"3fe29758-690b-4ea7-83af-b2c70b9db594\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:28.178008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.177989 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78248e42-57ee-4ed2-bbe4-928af2067507-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbqhw\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.195008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.194940 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxq56\" (UniqueName: \"kubernetes.io/projected/78248e42-57ee-4ed2-bbe4-928af2067507-kube-api-access-zxq56\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbqhw\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.225232 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.225209 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.227761 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.227729 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.229550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.229536 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.229902 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.229865 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.234284 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.234265 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.236106 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.236088 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.278596 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.278565 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt6t4\" (UniqueName: \"kubernetes.io/projected/e07cd1c0-8cc2-49b6-b97d-7a733a274331-kube-api-access-gt6t4\") pod \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " Jul 01 23:12:28.278753 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.278682 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fk5c\" (UniqueName: \"kubernetes.io/projected/27c4236a-5a70-4009-9617-b8f7dbb7d915-kube-api-access-7fk5c\") pod \"27c4236a-5a70-4009-9617-b8f7dbb7d915\" (UID: \"27c4236a-5a70-4009-9617-b8f7dbb7d915\") " Jul 01 23:12:28.278753 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.278710 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e07cd1c0-8cc2-49b6-b97d-7a733a274331-extensions-socket-volume\") pod \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\" (UID: \"e07cd1c0-8cc2-49b6-b97d-7a733a274331\") " Jul 01 23:12:28.278965 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.278941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5tfl\" (UniqueName: \"kubernetes.io/projected/3fe29758-690b-4ea7-83af-b2c70b9db594-kube-api-access-k5tfl\") pod \"limitador-operator-controller-manager-859d7b78bd-f4b8q\" (UID: \"3fe29758-690b-4ea7-83af-b2c70b9db594\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:28.279253 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.279226 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07cd1c0-8cc2-49b6-b97d-7a733a274331-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e07cd1c0-8cc2-49b6-b97d-7a733a274331" (UID: "e07cd1c0-8cc2-49b6-b97d-7a733a274331"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:12:28.280694 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.280671 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e07cd1c0-8cc2-49b6-b97d-7a733a274331-kube-api-access-gt6t4" (OuterVolumeSpecName: "kube-api-access-gt6t4") pod "e07cd1c0-8cc2-49b6-b97d-7a733a274331" (UID: "e07cd1c0-8cc2-49b6-b97d-7a733a274331"). InnerVolumeSpecName "kube-api-access-gt6t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:12:28.280778 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.280754 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c4236a-5a70-4009-9617-b8f7dbb7d915-kube-api-access-7fk5c" (OuterVolumeSpecName: "kube-api-access-7fk5c") pod "27c4236a-5a70-4009-9617-b8f7dbb7d915" (UID: "27c4236a-5a70-4009-9617-b8f7dbb7d915"). InnerVolumeSpecName "kube-api-access-7fk5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:12:28.288997 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.288972 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5tfl\" (UniqueName: \"kubernetes.io/projected/3fe29758-690b-4ea7-83af-b2c70b9db594-kube-api-access-k5tfl\") pod \"limitador-operator-controller-manager-859d7b78bd-f4b8q\" (UID: \"3fe29758-690b-4ea7-83af-b2c70b9db594\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:28.313047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.313005 2569 generic.go:356] "Generic (PLEG): container finished" podID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" containerID="34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0" exitCode=0 Jul 01 23:12:28.313159 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.313053 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.313159 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.313114 2569 scope.go:117] "RemoveContainer" containerID="34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0" Jul 01 23:12:28.314518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.314488 2569 generic.go:356] "Generic (PLEG): container finished" podID="27c4236a-5a70-4009-9617-b8f7dbb7d915" containerID="446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61" exitCode=0 Jul 01 23:12:28.314644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.314537 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.314712 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.314687 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:28.315532 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.315505 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.317637 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.317588 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.319494 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.319470 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:28.321192 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.321169 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.322871 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.322847 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.323450 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.323435 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:28.324473 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.324452 2569 scope.go:117] "RemoveContainer" containerID="34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0" Jul 01 23:12:28.324754 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.324736 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0\": container with ID starting with 34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0 not found: ID does not exist" containerID="34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0" Jul 01 23:12:28.324825 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.324765 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0"} err="failed to get container status \"34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0\": rpc error: code = NotFound desc = could not find container \"34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0\": container with ID starting with 34b7782076fe07496a687a4c695310f2beb80864eb93e2d503894da603f6abd0 not found: ID does not exist" Jul 01 23:12:28.324825 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.324788 2569 scope.go:117] "RemoveContainer" containerID="446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61" Jul 01 23:12:28.325248 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.325226 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.327219 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.327195 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.329010 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.328976 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:28.330700 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.330674 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95ghp\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95ghp" Jul 01 23:12:28.332274 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.332250 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lt8c5\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lt8c5" Jul 01 23:12:28.333221 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.333203 2569 scope.go:117] "RemoveContainer" containerID="446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61" Jul 01 23:12:28.333452 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.333432 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61\": container with ID starting with 446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61 not found: ID does not exist" containerID="446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61" Jul 01 23:12:28.333522 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.333458 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61"} err="failed to get container status \"446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61\": rpc error: code = NotFound desc = could not find container \"446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61\": container with ID starting with 446d6edbf39aa69b374a0575c26697bd292741b60e954116f33ddec605515f61 not found: ID does not exist" Jul 01 23:12:28.334085 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:28.334070 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:28.379443 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379411 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ceb73e4-19fc-4cba-b68c-55616bf20658-extensions-socket-volume\") pod \"3ceb73e4-19fc-4cba-b68c-55616bf20658\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " Jul 01 23:12:28.379781 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379466 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x8bj\" (UniqueName: \"kubernetes.io/projected/3ceb73e4-19fc-4cba-b68c-55616bf20658-kube-api-access-8x8bj\") pod \"3ceb73e4-19fc-4cba-b68c-55616bf20658\" (UID: \"3ceb73e4-19fc-4cba-b68c-55616bf20658\") " Jul 01 23:12:28.379781 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379679 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ceb73e4-19fc-4cba-b68c-55616bf20658-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3ceb73e4-19fc-4cba-b68c-55616bf20658" (UID: "3ceb73e4-19fc-4cba-b68c-55616bf20658"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:12:28.379856 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379815 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt6t4\" (UniqueName: \"kubernetes.io/projected/e07cd1c0-8cc2-49b6-b97d-7a733a274331-kube-api-access-gt6t4\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:28.379856 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379827 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ceb73e4-19fc-4cba-b68c-55616bf20658-extensions-socket-volume\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:28.379856 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379837 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fk5c\" (UniqueName: \"kubernetes.io/projected/27c4236a-5a70-4009-9617-b8f7dbb7d915-kube-api-access-7fk5c\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:28.379856 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.379845 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e07cd1c0-8cc2-49b6-b97d-7a733a274331-extensions-socket-volume\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:28.381401 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.381377 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ceb73e4-19fc-4cba-b68c-55616bf20658-kube-api-access-8x8bj" (OuterVolumeSpecName: "kube-api-access-8x8bj") pod "3ceb73e4-19fc-4cba-b68c-55616bf20658" (UID: "3ceb73e4-19fc-4cba-b68c-55616bf20658"). InnerVolumeSpecName "kube-api-access-8x8bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:12:28.391556 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.391538 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:28.397688 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.397625 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:28.481017 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.480966 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x8bj\" (UniqueName: \"kubernetes.io/projected/3ceb73e4-19fc-4cba-b68c-55616bf20658-kube-api-access-8x8bj\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:28.557601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.557575 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw"] Jul 01 23:12:28.559776 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:28.559740 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78248e42_57ee_4ed2_bbe4_928af2067507.slice/crio-c2efc814e678e753d3e2dcd500bc2fdee2940dae8dbc000c56a0011dceb8fefd WatchSource:0}: Error finding container c2efc814e678e753d3e2dcd500bc2fdee2940dae8dbc000c56a0011dceb8fefd: Status 404 returned error can't find the container with id c2efc814e678e753d3e2dcd500bc2fdee2940dae8dbc000c56a0011dceb8fefd Jul 01 23:12:28.592072 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.591971 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q"] Jul 01 23:12:28.595088 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:28.595063 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe29758_690b_4ea7_83af_b2c70b9db594.slice/crio-0d451ae4ac29148db55aaac20f5857507e9b4da977c6acb3ead903ce1dcc8560 WatchSource:0}: Error finding container 0d451ae4ac29148db55aaac20f5857507e9b4da977c6acb3ead903ce1dcc8560: Status 404 returned error can't find the container with id 0d451ae4ac29148db55aaac20f5857507e9b4da977c6acb3ead903ce1dcc8560 Jul 01 23:12:28.923206 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.923174 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c4236a-5a70-4009-9617-b8f7dbb7d915" path="/var/lib/kubelet/pods/27c4236a-5a70-4009-9617-b8f7dbb7d915/volumes" Jul 01 23:12:28.923522 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.923509 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" path="/var/lib/kubelet/pods/3ceb73e4-19fc-4cba-b68c-55616bf20658/volumes" Jul 01 23:12:28.923758 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:28.923746 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e07cd1c0-8cc2-49b6-b97d-7a733a274331" path="/var/lib/kubelet/pods/e07cd1c0-8cc2-49b6-b97d-7a733a274331/volumes" Jul 01 23:12:29.265101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.265022 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qb9ht" Jul 01 23:12:29.324553 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.324519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" event={"ID":"3fe29758-690b-4ea7-83af-b2c70b9db594","Type":"ContainerStarted","Data":"c982fc3bc49171bec74d8ee067a1d86211de005785530e1c5d58d9bea03787cf"} Jul 01 23:12:29.324741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.324579 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" event={"ID":"3fe29758-690b-4ea7-83af-b2c70b9db594","Type":"ContainerStarted","Data":"0d451ae4ac29148db55aaac20f5857507e9b4da977c6acb3ead903ce1dcc8560"} Jul 01 23:12:29.324741 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.324636 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:29.326576 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.326542 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" event={"ID":"78248e42-57ee-4ed2-bbe4-928af2067507","Type":"ContainerStarted","Data":"64652bd5e53cd3d89a8ef1de254a7295b7aae4b05316f2ad2bd43b78c07b67d2"} Jul 01 23:12:29.326789 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.326595 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:29.326789 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.326601 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:29.326789 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.326631 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" event={"ID":"78248e42-57ee-4ed2-bbe4-928af2067507","Type":"ContainerStarted","Data":"c2efc814e678e753d3e2dcd500bc2fdee2940dae8dbc000c56a0011dceb8fefd"} Jul 01 23:12:29.350292 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.350240 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" podStartSLOduration=2.350221415 podStartE2EDuration="2.350221415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:29.350207037 +0000 UTC m=+479.572735271" watchObservedRunningTime="2026-07-01 23:12:29.350221415 +0000 UTC m=+479.572749626" Jul 01 23:12:29.381142 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:29.381093 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" podStartSLOduration=2.381079144 podStartE2EDuration="2.381079144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:29.378746329 +0000 UTC m=+479.601274550" watchObservedRunningTime="2026-07-01 23:12:29.381079144 +0000 UTC m=+479.603607353" Jul 01 23:12:29.385930 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:29.385904 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:30.922200 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:30.922160 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-xtv76\" is forbidden: User \"system:node:ip-10-0-141-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-234.ec2.internal' and this object" podUID="3ceb73e4-19fc-4cba-b68c-55616bf20658" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xtv76" Jul 01 23:12:40.337077 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:40.337043 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:40.337501 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:40.337377 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f4b8q" Jul 01 23:12:45.295645 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.295557 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw"] Jul 01 23:12:45.296088 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.295907 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" podUID="78248e42-57ee-4ed2-bbe4-928af2067507" containerName="manager" containerID="cri-o://64652bd5e53cd3d89a8ef1de254a7295b7aae4b05316f2ad2bd43b78c07b67d2" gracePeriod=10 Jul 01 23:12:45.418650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.418618 2569 generic.go:356] "Generic (PLEG): container finished" podID="78248e42-57ee-4ed2-bbe4-928af2067507" containerID="64652bd5e53cd3d89a8ef1de254a7295b7aae4b05316f2ad2bd43b78c07b67d2" exitCode=0 Jul 01 23:12:45.418650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.418638 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" event={"ID":"78248e42-57ee-4ed2-bbe4-928af2067507","Type":"ContainerDied","Data":"64652bd5e53cd3d89a8ef1de254a7295b7aae4b05316f2ad2bd43b78c07b67d2"} Jul 01 23:12:45.548623 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.548553 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:45.585118 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.585089 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj"] Jul 01 23:12:45.585512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.585500 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78248e42-57ee-4ed2-bbe4-928af2067507" containerName="manager" Jul 01 23:12:45.585559 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.585514 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="78248e42-57ee-4ed2-bbe4-928af2067507" containerName="manager" Jul 01 23:12:45.585593 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.585585 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="78248e42-57ee-4ed2-bbe4-928af2067507" containerName="manager" Jul 01 23:12:45.588979 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.588964 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.604772 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.604745 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj"] Jul 01 23:12:45.627582 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.627553 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78248e42-57ee-4ed2-bbe4-928af2067507-extensions-socket-volume\") pod \"78248e42-57ee-4ed2-bbe4-928af2067507\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " Jul 01 23:12:45.627694 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.627658 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxq56\" (UniqueName: \"kubernetes.io/projected/78248e42-57ee-4ed2-bbe4-928af2067507-kube-api-access-zxq56\") pod \"78248e42-57ee-4ed2-bbe4-928af2067507\" (UID: \"78248e42-57ee-4ed2-bbe4-928af2067507\") " Jul 01 23:12:45.627912 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.627892 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78248e42-57ee-4ed2-bbe4-928af2067507-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "78248e42-57ee-4ed2-bbe4-928af2067507" (UID: "78248e42-57ee-4ed2-bbe4-928af2067507"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:12:45.629619 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.629583 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78248e42-57ee-4ed2-bbe4-928af2067507-kube-api-access-zxq56" (OuterVolumeSpecName: "kube-api-access-zxq56") pod "78248e42-57ee-4ed2-bbe4-928af2067507" (UID: "78248e42-57ee-4ed2-bbe4-928af2067507"). InnerVolumeSpecName "kube-api-access-zxq56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:12:45.728560 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.728524 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzpbm\" (UniqueName: \"kubernetes.io/projected/9a980a54-77ac-4443-93df-0679af367725-kube-api-access-bzpbm\") pod \"kuadrant-operator-controller-manager-86878c769f-42mrj\" (UID: \"9a980a54-77ac-4443-93df-0679af367725\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.728560 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.728565 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a980a54-77ac-4443-93df-0679af367725-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42mrj\" (UID: \"9a980a54-77ac-4443-93df-0679af367725\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.728771 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.728724 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxq56\" (UniqueName: \"kubernetes.io/projected/78248e42-57ee-4ed2-bbe4-928af2067507-kube-api-access-zxq56\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:45.728771 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.728742 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78248e42-57ee-4ed2-bbe4-928af2067507-extensions-socket-volume\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:45.829766 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.829738 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzpbm\" (UniqueName: \"kubernetes.io/projected/9a980a54-77ac-4443-93df-0679af367725-kube-api-access-bzpbm\") pod \"kuadrant-operator-controller-manager-86878c769f-42mrj\" (UID: \"9a980a54-77ac-4443-93df-0679af367725\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.829898 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.829775 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a980a54-77ac-4443-93df-0679af367725-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42mrj\" (UID: \"9a980a54-77ac-4443-93df-0679af367725\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.830172 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.830155 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a980a54-77ac-4443-93df-0679af367725-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-42mrj\" (UID: \"9a980a54-77ac-4443-93df-0679af367725\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.839528 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.839501 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzpbm\" (UniqueName: \"kubernetes.io/projected/9a980a54-77ac-4443-93df-0679af367725-kube-api-access-bzpbm\") pod \"kuadrant-operator-controller-manager-86878c769f-42mrj\" (UID: \"9a980a54-77ac-4443-93df-0679af367725\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:45.900239 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:45.900212 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:46.038568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.038543 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj"] Jul 01 23:12:46.040452 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:12:46.040431 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a980a54_77ac_4443_93df_0679af367725.slice/crio-19b0a2f0a25a3345355a23ae638fee50fd3eeb3504c54e75de0abb88d833e3bc WatchSource:0}: Error finding container 19b0a2f0a25a3345355a23ae638fee50fd3eeb3504c54e75de0abb88d833e3bc: Status 404 returned error can't find the container with id 19b0a2f0a25a3345355a23ae638fee50fd3eeb3504c54e75de0abb88d833e3bc Jul 01 23:12:46.426355 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.426277 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" Jul 01 23:12:46.426355 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.426288 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw" event={"ID":"78248e42-57ee-4ed2-bbe4-928af2067507","Type":"ContainerDied","Data":"c2efc814e678e753d3e2dcd500bc2fdee2940dae8dbc000c56a0011dceb8fefd"} Jul 01 23:12:46.426355 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.426354 2569 scope.go:117] "RemoveContainer" containerID="64652bd5e53cd3d89a8ef1de254a7295b7aae4b05316f2ad2bd43b78c07b67d2" Jul 01 23:12:46.428194 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.428171 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" event={"ID":"9a980a54-77ac-4443-93df-0679af367725","Type":"ContainerStarted","Data":"cf532c73b9960ad75c6d1759a64ab033751fda927d5739910afc2f0ea87757d2"} Jul 01 23:12:46.428298 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.428211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" event={"ID":"9a980a54-77ac-4443-93df-0679af367725","Type":"ContainerStarted","Data":"19b0a2f0a25a3345355a23ae638fee50fd3eeb3504c54e75de0abb88d833e3bc"} Jul 01 23:12:46.428356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.428322 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:12:46.471168 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.471138 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw"] Jul 01 23:12:46.478277 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.478255 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbqhw"] Jul 01 23:12:46.514852 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.514807 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" podStartSLOduration=1.5147943430000002 podStartE2EDuration="1.514794343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:12:46.512945346 +0000 UTC m=+496.735473571" watchObservedRunningTime="2026-07-01 23:12:46.514794343 +0000 UTC m=+496.737322552" Jul 01 23:12:46.923186 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:46.923143 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78248e42-57ee-4ed2-bbe4-928af2067507" path="/var/lib/kubelet/pods/78248e42-57ee-4ed2-bbe4-928af2067507/volumes" Jul 01 23:12:51.400410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.400350 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d65b8d58d-slbgt" podUID="95789aac-e27e-47a6-818f-562c2d53e765" containerName="console" containerID="cri-o://b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d" gracePeriod=15 Jul 01 23:12:51.646517 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.646495 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:12:51.780827 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780747 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-oauth-config\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.780827 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780778 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-console-config\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.780827 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780814 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdsfk\" (UniqueName: \"kubernetes.io/projected/95789aac-e27e-47a6-818f-562c2d53e765-kube-api-access-tdsfk\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.781087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780839 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-oauth-serving-cert\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.781087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780886 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-service-ca\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.781087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780910 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-serving-cert\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.781087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.780959 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-trusted-ca-bundle\") pod \"95789aac-e27e-47a6-818f-562c2d53e765\" (UID: \"95789aac-e27e-47a6-818f-562c2d53e765\") " Jul 01 23:12:51.781306 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.781239 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-console-config" (OuterVolumeSpecName: "console-config") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:12:51.781364 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.781344 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:12:51.781405 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.781358 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-service-ca" (OuterVolumeSpecName: "service-ca") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:12:51.781698 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.781662 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:12:51.782929 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.782901 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:12:51.783036 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.783017 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95789aac-e27e-47a6-818f-562c2d53e765-kube-api-access-tdsfk" (OuterVolumeSpecName: "kube-api-access-tdsfk") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "kube-api-access-tdsfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:12:51.783036 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.783019 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "95789aac-e27e-47a6-818f-562c2d53e765" (UID: "95789aac-e27e-47a6-818f-562c2d53e765"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:12:51.881857 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881827 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-oauth-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:51.881857 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881854 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-console-config\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:51.882031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881868 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdsfk\" (UniqueName: \"kubernetes.io/projected/95789aac-e27e-47a6-818f-562c2d53e765-kube-api-access-tdsfk\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:51.882031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881881 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-oauth-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:51.882031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881892 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-service-ca\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:51.882031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881904 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95789aac-e27e-47a6-818f-562c2d53e765-console-serving-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:51.882031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:51.881918 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95789aac-e27e-47a6-818f-562c2d53e765-trusted-ca-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:12:52.462638 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.462580 2569 generic.go:356] "Generic (PLEG): container finished" podID="95789aac-e27e-47a6-818f-562c2d53e765" containerID="b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d" exitCode=0 Jul 01 23:12:52.463050 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.462645 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d65b8d58d-slbgt" Jul 01 23:12:52.463050 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.462659 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d65b8d58d-slbgt" event={"ID":"95789aac-e27e-47a6-818f-562c2d53e765","Type":"ContainerDied","Data":"b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d"} Jul 01 23:12:52.463050 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.462710 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d65b8d58d-slbgt" event={"ID":"95789aac-e27e-47a6-818f-562c2d53e765","Type":"ContainerDied","Data":"e2500b3240f87ff920a66671b1608946b657055b08277ee4b0e3624c37b7860a"} Jul 01 23:12:52.463050 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.462739 2569 scope.go:117] "RemoveContainer" containerID="b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d" Jul 01 23:12:52.475333 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.475305 2569 scope.go:117] "RemoveContainer" containerID="b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d" Jul 01 23:12:52.475711 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:12:52.475681 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d\": container with ID starting with b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d not found: ID does not exist" containerID="b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d" Jul 01 23:12:52.475807 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.475725 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d"} err="failed to get container status \"b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d\": rpc error: code = NotFound desc = could not find container \"b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d\": container with ID starting with b0cd3c80a66239a99196b6dca3b92ca919344f3ba837b201086b99765208866d not found: ID does not exist" Jul 01 23:12:52.492407 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.492374 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d65b8d58d-slbgt"] Jul 01 23:12:52.499048 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.499028 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d65b8d58d-slbgt"] Jul 01 23:12:52.924141 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:52.924106 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95789aac-e27e-47a6-818f-562c2d53e765" path="/var/lib/kubelet/pods/95789aac-e27e-47a6-818f-562c2d53e765/volumes" Jul 01 23:12:57.437057 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:12:57.437029 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-42mrj" Jul 01 23:13:01.822118 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.822084 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x"] Jul 01 23:13:01.822514 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.822493 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95789aac-e27e-47a6-818f-562c2d53e765" containerName="console" Jul 01 23:13:01.822514 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.822505 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="95789aac-e27e-47a6-818f-562c2d53e765" containerName="console" Jul 01 23:13:01.822616 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.822590 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="95789aac-e27e-47a6-818f-562c2d53e765" containerName="console" Jul 01 23:13:01.826016 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.825997 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.830518 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.830500 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dnc9t\"" Jul 01 23:13:01.831016 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.830996 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 01 23:13:01.842049 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.842024 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x"] Jul 01 23:13:01.862957 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.862928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.862957 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.862958 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.862992 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.863037 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863127 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.863093 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863238 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.863148 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863238 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.863167 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863238 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.863185 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zqsd\" (UniqueName: \"kubernetes.io/projected/0283a6bf-b13e-4dd3-a602-61cf1d715db8-kube-api-access-7zqsd\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.863238 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.863209 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.963697 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963650 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.963888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963816 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.963888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963853 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.963888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963877 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zqsd\" (UniqueName: \"kubernetes.io/projected/0283a6bf-b13e-4dd3-a602-61cf1d715db8-kube-api-access-7zqsd\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964005 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963915 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964005 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964005 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.963979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964153 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964278 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964187 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964283 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964412 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964373 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964472 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964458 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.964558 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.964534 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.966251 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.966226 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.966498 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.966476 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.973577 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.973558 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0283a6bf-b13e-4dd3-a602-61cf1d715db8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:01.982327 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:01.982300 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zqsd\" (UniqueName: \"kubernetes.io/projected/0283a6bf-b13e-4dd3-a602-61cf1d715db8-kube-api-access-7zqsd\") pod \"maas-default-gateway-openshift-default-56654dcf7-b5l2x\" (UID: \"0283a6bf-b13e-4dd3-a602-61cf1d715db8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:02.136899 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:02.136801 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:02.286695 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:02.286665 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x"] Jul 01 23:13:02.288555 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:13:02.288532 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0283a6bf_b13e_4dd3_a602_61cf1d715db8.slice/crio-2382155bd08937278ea79eb2d32157b81260a35a716e6c52bc5cfef8b79c390d WatchSource:0}: Error finding container 2382155bd08937278ea79eb2d32157b81260a35a716e6c52bc5cfef8b79c390d: Status 404 returned error can't find the container with id 2382155bd08937278ea79eb2d32157b81260a35a716e6c52bc5cfef8b79c390d Jul 01 23:13:02.520143 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:02.520050 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" event={"ID":"0283a6bf-b13e-4dd3-a602-61cf1d715db8","Type":"ContainerStarted","Data":"2382155bd08937278ea79eb2d32157b81260a35a716e6c52bc5cfef8b79c390d"} Jul 01 23:13:05.782040 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:05.782004 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:13:05.782294 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:05.782075 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:13:05.782294 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:05.782106 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:13:06.546745 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:06.546711 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" event={"ID":"0283a6bf-b13e-4dd3-a602-61cf1d715db8","Type":"ContainerStarted","Data":"121eb763875dfe85dad69fa02e9a8bc35ddd86096954cbdbf5285df9d266236b"} Jul 01 23:13:06.573165 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:06.573113 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" podStartSLOduration=2.081849623 podStartE2EDuration="5.573096545s" totalImagesPullingTime="3.491246922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:13:06.570078026 +0000 UTC m=+516.792606262" watchObservedRunningTime="2026-07-01 23:13:06.573096545 +0000 UTC m=+516.795624856" Jul 01 23:13:07.137351 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:07.137316 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:07.142111 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:07.142088 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:07.552920 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:07.552833 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:07.553889 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:07.553870 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:13:17.289383 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.289346 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:13:17.293437 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.293417 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.295847 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.295818 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 01 23:13:17.295948 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.295847 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kbgjj\"" Jul 01 23:13:17.305100 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.305077 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:13:17.339373 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.339347 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:13:17.402874 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.402848 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srgxj\" (UniqueName: \"kubernetes.io/projected/fa57adf9-9295-432c-871b-f1dd0313d4ca-kube-api-access-srgxj\") pod \"limitador-limitador-54d6d988d5-gc8l6\" (UID: \"fa57adf9-9295-432c-871b-f1dd0313d4ca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.403008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.402908 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa57adf9-9295-432c-871b-f1dd0313d4ca-config-file\") pod \"limitador-limitador-54d6d988d5-gc8l6\" (UID: \"fa57adf9-9295-432c-871b-f1dd0313d4ca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.503623 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.503583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa57adf9-9295-432c-871b-f1dd0313d4ca-config-file\") pod \"limitador-limitador-54d6d988d5-gc8l6\" (UID: \"fa57adf9-9295-432c-871b-f1dd0313d4ca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.503782 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.503708 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srgxj\" (UniqueName: \"kubernetes.io/projected/fa57adf9-9295-432c-871b-f1dd0313d4ca-kube-api-access-srgxj\") pod \"limitador-limitador-54d6d988d5-gc8l6\" (UID: \"fa57adf9-9295-432c-871b-f1dd0313d4ca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.504216 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.504195 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa57adf9-9295-432c-871b-f1dd0313d4ca-config-file\") pod \"limitador-limitador-54d6d988d5-gc8l6\" (UID: \"fa57adf9-9295-432c-871b-f1dd0313d4ca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.511869 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.511845 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srgxj\" (UniqueName: \"kubernetes.io/projected/fa57adf9-9295-432c-871b-f1dd0313d4ca-kube-api-access-srgxj\") pod \"limitador-limitador-54d6d988d5-gc8l6\" (UID: \"fa57adf9-9295-432c-871b-f1dd0313d4ca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.605319 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.605282 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:17.747087 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.747047 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:13:17.750085 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:13:17.750053 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa57adf9_9295_432c_871b_f1dd0313d4ca.slice/crio-bb81a6b4a905fc9f159ea293eec9368faa0f76ef5844e419ee6edc1afb00737b WatchSource:0}: Error finding container bb81a6b4a905fc9f159ea293eec9368faa0f76ef5844e419ee6edc1afb00737b: Status 404 returned error can't find the container with id bb81a6b4a905fc9f159ea293eec9368faa0f76ef5844e419ee6edc1afb00737b Jul 01 23:13:17.860340 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.860261 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hdn7c"] Jul 01 23:13:17.865183 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.865168 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:13:17.867588 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.867568 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t64dd\"" Jul 01 23:13:17.869209 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.869182 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hdn7c"] Jul 01 23:13:17.907727 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:17.907695 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb762\" (UniqueName: \"kubernetes.io/projected/cff243eb-ce2a-4167-bd1c-792d39f6f774-kube-api-access-lb762\") pod \"authorino-75d556bd77-hdn7c\" (UID: \"cff243eb-ce2a-4167-bd1c-792d39f6f774\") " pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:13:18.008371 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:18.008340 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb762\" (UniqueName: \"kubernetes.io/projected/cff243eb-ce2a-4167-bd1c-792d39f6f774-kube-api-access-lb762\") pod \"authorino-75d556bd77-hdn7c\" (UID: \"cff243eb-ce2a-4167-bd1c-792d39f6f774\") " pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:13:18.016951 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:18.016923 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb762\" (UniqueName: \"kubernetes.io/projected/cff243eb-ce2a-4167-bd1c-792d39f6f774-kube-api-access-lb762\") pod \"authorino-75d556bd77-hdn7c\" (UID: \"cff243eb-ce2a-4167-bd1c-792d39f6f774\") " pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:13:18.176885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:18.176807 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:13:18.328509 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:18.328470 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hdn7c"] Jul 01 23:13:18.333075 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:13:18.333046 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcff243eb_ce2a_4167_bd1c_792d39f6f774.slice/crio-e3ec9b788a62db3733315635653b0e0c84dacc77e900e2ee3ae8ea4df2824d15 WatchSource:0}: Error finding container e3ec9b788a62db3733315635653b0e0c84dacc77e900e2ee3ae8ea4df2824d15: Status 404 returned error can't find the container with id e3ec9b788a62db3733315635653b0e0c84dacc77e900e2ee3ae8ea4df2824d15 Jul 01 23:13:18.621818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:18.621782 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hdn7c" event={"ID":"cff243eb-ce2a-4167-bd1c-792d39f6f774","Type":"ContainerStarted","Data":"e3ec9b788a62db3733315635653b0e0c84dacc77e900e2ee3ae8ea4df2824d15"} Jul 01 23:13:18.623261 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:18.623234 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" event={"ID":"fa57adf9-9295-432c-871b-f1dd0313d4ca","Type":"ContainerStarted","Data":"bb81a6b4a905fc9f159ea293eec9368faa0f76ef5844e419ee6edc1afb00737b"} Jul 01 23:13:22.652913 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:22.652874 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hdn7c" event={"ID":"cff243eb-ce2a-4167-bd1c-792d39f6f774","Type":"ContainerStarted","Data":"737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630"} Jul 01 23:13:22.654473 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:22.654448 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" event={"ID":"fa57adf9-9295-432c-871b-f1dd0313d4ca","Type":"ContainerStarted","Data":"7c4a3e72d145632216df2704aa42009f28124a172c6e21cb10a43158eaf822bf"} Jul 01 23:13:22.654600 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:22.654543 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:22.668460 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:22.668412 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hdn7c" podStartSLOduration=1.7000622399999998 podStartE2EDuration="5.668397935s" totalImagesPullingTime="3.968335695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:13:22.667917122 +0000 UTC m=+532.890445335" watchObservedRunningTime="2026-07-01 23:13:22.668397935 +0000 UTC m=+532.890926144" Jul 01 23:13:22.686149 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:22.686103 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" podStartSLOduration=1.134643329 podStartE2EDuration="5.686087988s" totalImagesPullingTime="4.551444659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:13:22.684179256 +0000 UTC m=+532.906707469" watchObservedRunningTime="2026-07-01 23:13:22.686087988 +0000 UTC m=+532.908616198" Jul 01 23:13:33.660447 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:33.660418 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gc8l6" Jul 01 23:13:52.961033 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:52.960998 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff"] Jul 01 23:13:53.017171 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.017089 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff"] Jul 01 23:13:53.017333 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.017200 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.020034 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.019991 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 01 23:13:53.020194 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.020095 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6j5mm\"" Jul 01 23:13:53.020248 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.020196 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 01 23:13:53.026317 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.026292 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.026395 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.026382 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.026441 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.026403 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh68q\" (UniqueName: \"kubernetes.io/projected/842220ac-6d79-449a-8cbf-4c71dccd283f-kube-api-access-hh68q\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.127857 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.127820 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.128052 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.127904 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.128052 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.127932 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh68q\" (UniqueName: \"kubernetes.io/projected/842220ac-6d79-449a-8cbf-4c71dccd283f-kube-api-access-hh68q\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.128221 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.128196 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.128281 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.128224 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.137415 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.137390 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh68q\" (UniqueName: \"kubernetes.io/projected/842220ac-6d79-449a-8cbf-4c71dccd283f-kube-api-access-hh68q\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.326865 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.326833 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:53.493163 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.493139 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff"] Jul 01 23:13:53.495885 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:13:53.495861 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842220ac_6d79_449a_8cbf_4c71dccd283f.slice/crio-dc430703d3e20b5495273bfc161d6a36d66f6d6128dd782c956f745ba5bb669f WatchSource:0}: Error finding container dc430703d3e20b5495273bfc161d6a36d66f6d6128dd782c956f745ba5bb669f: Status 404 returned error can't find the container with id dc430703d3e20b5495273bfc161d6a36d66f6d6128dd782c956f745ba5bb669f Jul 01 23:13:53.835305 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.835274 2569 generic.go:356] "Generic (PLEG): container finished" podID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerID="628d8673b3f4e82391ac21781093ee05b92736bdb8e7f1c429f246aa4e6b6a88" exitCode=0 Jul 01 23:13:53.835485 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.835319 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" event={"ID":"842220ac-6d79-449a-8cbf-4c71dccd283f","Type":"ContainerDied","Data":"628d8673b3f4e82391ac21781093ee05b92736bdb8e7f1c429f246aa4e6b6a88"} Jul 01 23:13:53.835485 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:53.835355 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" event={"ID":"842220ac-6d79-449a-8cbf-4c71dccd283f","Type":"ContainerStarted","Data":"dc430703d3e20b5495273bfc161d6a36d66f6d6128dd782c956f745ba5bb669f"} Jul 01 23:13:55.850041 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:55.850008 2569 generic.go:356] "Generic (PLEG): container finished" podID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerID="13ba269e75f15c94dcbbf7c68c869781c98eafab9ad0040c1bb250cf5f8e4210" exitCode=0 Jul 01 23:13:55.850519 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:55.850074 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" event={"ID":"842220ac-6d79-449a-8cbf-4c71dccd283f","Type":"ContainerDied","Data":"13ba269e75f15c94dcbbf7c68c869781c98eafab9ad0040c1bb250cf5f8e4210"} Jul 01 23:13:56.857633 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:56.857583 2569 generic.go:356] "Generic (PLEG): container finished" podID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerID="44f60be64d3e874025587c5b59aca154a94b4dd3ff517dec544683f4ebbb3145" exitCode=0 Jul 01 23:13:56.858101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:56.857637 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" event={"ID":"842220ac-6d79-449a-8cbf-4c71dccd283f","Type":"ContainerDied","Data":"44f60be64d3e874025587c5b59aca154a94b4dd3ff517dec544683f4ebbb3145"} Jul 01 23:13:58.002053 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.002028 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:58.069980 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.069947 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh68q\" (UniqueName: \"kubernetes.io/projected/842220ac-6d79-449a-8cbf-4c71dccd283f-kube-api-access-hh68q\") pod \"842220ac-6d79-449a-8cbf-4c71dccd283f\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " Jul 01 23:13:58.070147 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.070042 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-util\") pod \"842220ac-6d79-449a-8cbf-4c71dccd283f\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " Jul 01 23:13:58.070147 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.070070 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-bundle\") pod \"842220ac-6d79-449a-8cbf-4c71dccd283f\" (UID: \"842220ac-6d79-449a-8cbf-4c71dccd283f\") " Jul 01 23:13:58.070575 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.070546 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-bundle" (OuterVolumeSpecName: "bundle") pod "842220ac-6d79-449a-8cbf-4c71dccd283f" (UID: "842220ac-6d79-449a-8cbf-4c71dccd283f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:13:58.071967 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.071939 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842220ac-6d79-449a-8cbf-4c71dccd283f-kube-api-access-hh68q" (OuterVolumeSpecName: "kube-api-access-hh68q") pod "842220ac-6d79-449a-8cbf-4c71dccd283f" (UID: "842220ac-6d79-449a-8cbf-4c71dccd283f"). InnerVolumeSpecName "kube-api-access-hh68q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:13:58.078636 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.078595 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-util" (OuterVolumeSpecName: "util") pod "842220ac-6d79-449a-8cbf-4c71dccd283f" (UID: "842220ac-6d79-449a-8cbf-4c71dccd283f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 23:13:58.170934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.170857 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh68q\" (UniqueName: \"kubernetes.io/projected/842220ac-6d79-449a-8cbf-4c71dccd283f-kube-api-access-hh68q\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:13:58.170934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.170883 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-util\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:13:58.170934 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.170892 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842220ac-6d79-449a-8cbf-4c71dccd283f-bundle\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:13:58.870950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.870921 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" Jul 01 23:13:58.871138 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.870948 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8shnff" event={"ID":"842220ac-6d79-449a-8cbf-4c71dccd283f","Type":"ContainerDied","Data":"dc430703d3e20b5495273bfc161d6a36d66f6d6128dd782c956f745ba5bb669f"} Jul 01 23:13:58.871138 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:13:58.871003 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc430703d3e20b5495273bfc161d6a36d66f6d6128dd782c956f745ba5bb669f" Jul 01 23:14:30.296454 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:14:30.296424 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:14:30.297008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:14:30.296749 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:15:03.310366 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310284 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dcfbd"] Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310835 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="extract" Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310851 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="extract" Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310862 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="pull" Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310869 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="pull" Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310887 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="util" Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310892 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="util" Jul 01 23:15:03.312817 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.310967 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="842220ac-6d79-449a-8cbf-4c71dccd283f" containerName="extract" Jul 01 23:15:03.313846 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.313828 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:03.321696 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.321670 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dcfbd"] Jul 01 23:15:03.459089 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.459055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmrch\" (UniqueName: \"kubernetes.io/projected/346f0bfe-9cbe-4eca-ad88-31854d0bb048-kube-api-access-hmrch\") pod \"authorino-5d4b67c446-dcfbd\" (UID: \"346f0bfe-9cbe-4eca-ad88-31854d0bb048\") " pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:03.536364 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.536321 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dcfbd"] Jul 01 23:15:03.536697 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:15:03.536670 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hmrch], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-dcfbd" podUID="346f0bfe-9cbe-4eca-ad88-31854d0bb048" Jul 01 23:15:03.559729 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.559702 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmrch\" (UniqueName: \"kubernetes.io/projected/346f0bfe-9cbe-4eca-ad88-31854d0bb048-kube-api-access-hmrch\") pod \"authorino-5d4b67c446-dcfbd\" (UID: \"346f0bfe-9cbe-4eca-ad88-31854d0bb048\") " pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:03.567679 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.567657 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-864c7799c7-9s8x7"] Jul 01 23:15:03.569329 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.569307 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmrch\" (UniqueName: \"kubernetes.io/projected/346f0bfe-9cbe-4eca-ad88-31854d0bb048-kube-api-access-hmrch\") pod \"authorino-5d4b67c446-dcfbd\" (UID: \"346f0bfe-9cbe-4eca-ad88-31854d0bb048\") " pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:03.571639 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.571598 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:03.578347 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.578326 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-864c7799c7-9s8x7"] Jul 01 23:15:03.660706 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.660677 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4txdt\" (UniqueName: \"kubernetes.io/projected/172c5257-175a-4589-8c75-b765aea8e5b4-kube-api-access-4txdt\") pod \"authorino-864c7799c7-9s8x7\" (UID: \"172c5257-175a-4589-8c75-b765aea8e5b4\") " pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:03.752221 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.752187 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-864c7799c7-9s8x7"] Jul 01 23:15:03.752502 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:15:03.752483 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4txdt], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-864c7799c7-9s8x7" podUID="172c5257-175a-4589-8c75-b765aea8e5b4" Jul 01 23:15:03.761742 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.761711 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4txdt\" (UniqueName: \"kubernetes.io/projected/172c5257-175a-4589-8c75-b765aea8e5b4-kube-api-access-4txdt\") pod \"authorino-864c7799c7-9s8x7\" (UID: \"172c5257-175a-4589-8c75-b765aea8e5b4\") " pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:03.771345 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:03.771323 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4txdt\" (UniqueName: \"kubernetes.io/projected/172c5257-175a-4589-8c75-b765aea8e5b4-kube-api-access-4txdt\") pod \"authorino-864c7799c7-9s8x7\" (UID: \"172c5257-175a-4589-8c75-b765aea8e5b4\") " pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:04.246025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.245991 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:04.246210 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.245991 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:04.254300 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.254272 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:04.257909 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.257888 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:04.367860 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.367829 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4txdt\" (UniqueName: \"kubernetes.io/projected/172c5257-175a-4589-8c75-b765aea8e5b4-kube-api-access-4txdt\") pod \"172c5257-175a-4589-8c75-b765aea8e5b4\" (UID: \"172c5257-175a-4589-8c75-b765aea8e5b4\") " Jul 01 23:15:04.368309 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.367954 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmrch\" (UniqueName: \"kubernetes.io/projected/346f0bfe-9cbe-4eca-ad88-31854d0bb048-kube-api-access-hmrch\") pod \"346f0bfe-9cbe-4eca-ad88-31854d0bb048\" (UID: \"346f0bfe-9cbe-4eca-ad88-31854d0bb048\") " Jul 01 23:15:04.369954 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.369927 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346f0bfe-9cbe-4eca-ad88-31854d0bb048-kube-api-access-hmrch" (OuterVolumeSpecName: "kube-api-access-hmrch") pod "346f0bfe-9cbe-4eca-ad88-31854d0bb048" (UID: "346f0bfe-9cbe-4eca-ad88-31854d0bb048"). InnerVolumeSpecName "kube-api-access-hmrch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:15:04.370080 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.369989 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172c5257-175a-4589-8c75-b765aea8e5b4-kube-api-access-4txdt" (OuterVolumeSpecName: "kube-api-access-4txdt") pod "172c5257-175a-4589-8c75-b765aea8e5b4" (UID: "172c5257-175a-4589-8c75-b765aea8e5b4"). InnerVolumeSpecName "kube-api-access-4txdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:15:04.469659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.469592 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmrch\" (UniqueName: \"kubernetes.io/projected/346f0bfe-9cbe-4eca-ad88-31854d0bb048-kube-api-access-hmrch\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:15:04.469659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.469652 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4txdt\" (UniqueName: \"kubernetes.io/projected/172c5257-175a-4589-8c75-b765aea8e5b4-kube-api-access-4txdt\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:15:04.879887 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.879852 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hdn7c"] Jul 01 23:15:04.880107 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:04.880061 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hdn7c" podUID="cff243eb-ce2a-4167-bd1c-792d39f6f774" containerName="authorino" containerID="cri-o://737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630" gracePeriod=30 Jul 01 23:15:05.144047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.143981 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:15:05.253464 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253432 2569 generic.go:356] "Generic (PLEG): container finished" podID="cff243eb-ce2a-4167-bd1c-792d39f6f774" containerID="737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630" exitCode=0 Jul 01 23:15:05.253647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253476 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hdn7c" Jul 01 23:15:05.253647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hdn7c" event={"ID":"cff243eb-ce2a-4167-bd1c-792d39f6f774","Type":"ContainerDied","Data":"737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630"} Jul 01 23:15:05.253647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253574 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hdn7c" event={"ID":"cff243eb-ce2a-4167-bd1c-792d39f6f774","Type":"ContainerDied","Data":"e3ec9b788a62db3733315635653b0e0c84dacc77e900e2ee3ae8ea4df2824d15"} Jul 01 23:15:05.253647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253585 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dcfbd" Jul 01 23:15:05.253647 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253598 2569 scope.go:117] "RemoveContainer" containerID="737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630" Jul 01 23:15:05.253906 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.253850 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864c7799c7-9s8x7" Jul 01 23:15:05.263108 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.263086 2569 scope.go:117] "RemoveContainer" containerID="737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630" Jul 01 23:15:05.263401 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:15:05.263380 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630\": container with ID starting with 737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630 not found: ID does not exist" containerID="737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630" Jul 01 23:15:05.263472 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.263421 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630"} err="failed to get container status \"737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630\": rpc error: code = NotFound desc = could not find container \"737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630\": container with ID starting with 737f65c2241754fbc2d3efd506397f8289b646dad8171140548641b436361630 not found: ID does not exist" Jul 01 23:15:05.278018 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.277993 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb762\" (UniqueName: \"kubernetes.io/projected/cff243eb-ce2a-4167-bd1c-792d39f6f774-kube-api-access-lb762\") pod \"cff243eb-ce2a-4167-bd1c-792d39f6f774\" (UID: \"cff243eb-ce2a-4167-bd1c-792d39f6f774\") " Jul 01 23:15:05.280043 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.280015 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cff243eb-ce2a-4167-bd1c-792d39f6f774-kube-api-access-lb762" (OuterVolumeSpecName: "kube-api-access-lb762") pod "cff243eb-ce2a-4167-bd1c-792d39f6f774" (UID: "cff243eb-ce2a-4167-bd1c-792d39f6f774"). InnerVolumeSpecName "kube-api-access-lb762". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:15:05.285696 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.285667 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dcfbd"] Jul 01 23:15:05.291494 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.291474 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dcfbd"] Jul 01 23:15:05.315818 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.315789 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-864c7799c7-9s8x7"] Jul 01 23:15:05.326001 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.325960 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-864c7799c7-9s8x7"] Jul 01 23:15:05.379008 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.378979 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb762\" (UniqueName: \"kubernetes.io/projected/cff243eb-ce2a-4167-bd1c-792d39f6f774-kube-api-access-lb762\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:15:05.575783 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.575748 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hdn7c"] Jul 01 23:15:05.583826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:05.583801 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hdn7c"] Jul 01 23:15:06.923073 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:06.923036 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="172c5257-175a-4589-8c75-b765aea8e5b4" path="/var/lib/kubelet/pods/172c5257-175a-4589-8c75-b765aea8e5b4/volumes" Jul 01 23:15:06.923492 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:06.923264 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346f0bfe-9cbe-4eca-ad88-31854d0bb048" path="/var/lib/kubelet/pods/346f0bfe-9cbe-4eca-ad88-31854d0bb048/volumes" Jul 01 23:15:06.923492 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:06.923445 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cff243eb-ce2a-4167-bd1c-792d39f6f774" path="/var/lib/kubelet/pods/cff243eb-ce2a-4167-bd1c-792d39f6f774/volumes" Jul 01 23:15:09.181882 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.181849 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f89f798cf-vcxzg"] Jul 01 23:15:09.182321 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.182308 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cff243eb-ce2a-4167-bd1c-792d39f6f774" containerName="authorino" Jul 01 23:15:09.182368 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.182325 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cff243eb-ce2a-4167-bd1c-792d39f6f774" containerName="authorino" Jul 01 23:15:09.182409 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.182397 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cff243eb-ce2a-4167-bd1c-792d39f6f774" containerName="authorino" Jul 01 23:15:09.186929 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.186916 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.189899 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.189879 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m72hk\"" Jul 01 23:15:09.190041 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.189897 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 01 23:15:09.190041 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.189921 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 01 23:15:09.201492 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.201471 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f89f798cf-vcxzg"] Jul 01 23:15:09.314792 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.314763 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/0f59fc1c-db76-4155-84fc-d348c5c8bddf-kube-api-access-ld7hm\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.314792 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.314791 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.383545 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.383495 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77d44c68c8-4nv79"] Jul 01 23:15:09.386976 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.386961 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.404075 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.404050 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77d44c68c8-4nv79"] Jul 01 23:15:09.415498 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.415470 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/0f59fc1c-db76-4155-84fc-d348c5c8bddf-kube-api-access-ld7hm\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.415498 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.415498 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.415678 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:15:09.415664 2569 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 01 23:15:09.415723 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:15:09.415718 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert podName:0f59fc1c-db76-4155-84fc-d348c5c8bddf nodeName:}" failed. No retries permitted until 2026-07-01 23:15:09.915703508 +0000 UTC m=+640.138231696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert") pod "maas-controller-6f89f798cf-vcxzg" (UID: "0f59fc1c-db76-4155-84fc-d348c5c8bddf") : secret "maas-controller-webhook-cert" not found Jul 01 23:15:09.426361 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.426343 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/0f59fc1c-db76-4155-84fc-d348c5c8bddf-kube-api-access-ld7hm\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.516568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.516494 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfxr8\" (UniqueName: \"kubernetes.io/projected/d49e6287-a7c5-41f1-950e-f55dbaa31190-kube-api-access-tfxr8\") pod \"maas-controller-77d44c68c8-4nv79\" (UID: \"d49e6287-a7c5-41f1-950e-f55dbaa31190\") " pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.516568 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.516537 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d49e6287-a7c5-41f1-950e-f55dbaa31190-webhook-cert\") pod \"maas-controller-77d44c68c8-4nv79\" (UID: \"d49e6287-a7c5-41f1-950e-f55dbaa31190\") " pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.617425 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.617395 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfxr8\" (UniqueName: \"kubernetes.io/projected/d49e6287-a7c5-41f1-950e-f55dbaa31190-kube-api-access-tfxr8\") pod \"maas-controller-77d44c68c8-4nv79\" (UID: \"d49e6287-a7c5-41f1-950e-f55dbaa31190\") " pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.617561 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.617436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d49e6287-a7c5-41f1-950e-f55dbaa31190-webhook-cert\") pod \"maas-controller-77d44c68c8-4nv79\" (UID: \"d49e6287-a7c5-41f1-950e-f55dbaa31190\") " pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.619883 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.619862 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d49e6287-a7c5-41f1-950e-f55dbaa31190-webhook-cert\") pod \"maas-controller-77d44c68c8-4nv79\" (UID: \"d49e6287-a7c5-41f1-950e-f55dbaa31190\") " pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.626255 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.626230 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfxr8\" (UniqueName: \"kubernetes.io/projected/d49e6287-a7c5-41f1-950e-f55dbaa31190-kube-api-access-tfxr8\") pod \"maas-controller-77d44c68c8-4nv79\" (UID: \"d49e6287-a7c5-41f1-950e-f55dbaa31190\") " pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.698088 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.698069 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:09.838283 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.838258 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77d44c68c8-4nv79"] Jul 01 23:15:09.839897 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:15:09.839869 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49e6287_a7c5_41f1_950e_f55dbaa31190.slice/crio-1edbf0294b0ba1e40c960e2a1a0a319be90ef306d67c6ac94e400e042e3abeb9 WatchSource:0}: Error finding container 1edbf0294b0ba1e40c960e2a1a0a319be90ef306d67c6ac94e400e042e3abeb9: Status 404 returned error can't find the container with id 1edbf0294b0ba1e40c960e2a1a0a319be90ef306d67c6ac94e400e042e3abeb9 Jul 01 23:15:09.841245 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.841226 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 23:15:09.920146 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.920117 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:09.922463 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:09.922444 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert\") pod \"maas-controller-6f89f798cf-vcxzg\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:10.096920 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:10.096894 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:10.228708 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:10.228687 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f89f798cf-vcxzg"] Jul 01 23:15:10.230209 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:15:10.230183 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f59fc1c_db76_4155_84fc_d348c5c8bddf.slice/crio-34e57aaaf938a5b8c93f0f8e9efd1814851e6a1e9788ba131a6080337a6b9bc0 WatchSource:0}: Error finding container 34e57aaaf938a5b8c93f0f8e9efd1814851e6a1e9788ba131a6080337a6b9bc0: Status 404 returned error can't find the container with id 34e57aaaf938a5b8c93f0f8e9efd1814851e6a1e9788ba131a6080337a6b9bc0 Jul 01 23:15:10.283803 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:10.283776 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" event={"ID":"0f59fc1c-db76-4155-84fc-d348c5c8bddf","Type":"ContainerStarted","Data":"34e57aaaf938a5b8c93f0f8e9efd1814851e6a1e9788ba131a6080337a6b9bc0"} Jul 01 23:15:10.284970 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:10.284946 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77d44c68c8-4nv79" event={"ID":"d49e6287-a7c5-41f1-950e-f55dbaa31190","Type":"ContainerStarted","Data":"1edbf0294b0ba1e40c960e2a1a0a319be90ef306d67c6ac94e400e042e3abeb9"} Jul 01 23:15:13.309559 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:13.309524 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" event={"ID":"0f59fc1c-db76-4155-84fc-d348c5c8bddf","Type":"ContainerStarted","Data":"d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa"} Jul 01 23:15:13.310056 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:13.309591 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:13.311415 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:13.311389 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77d44c68c8-4nv79" event={"ID":"d49e6287-a7c5-41f1-950e-f55dbaa31190","Type":"ContainerStarted","Data":"3d2bb758d8a0ef27567c9347a0ca21f95156d1d9c12544e48efa1876717032ba"} Jul 01 23:15:13.311566 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:13.311514 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:13.329397 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:13.329335 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" podStartSLOduration=1.9765134309999999 podStartE2EDuration="4.329314423s" totalImagesPullingTime="2.352800992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:15:13.327185936 +0000 UTC m=+643.549714176" watchObservedRunningTime="2026-07-01 23:15:13.329314423 +0000 UTC m=+643.551842637" Jul 01 23:15:13.354302 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:13.354258 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77d44c68c8-4nv79" podStartSLOduration=1.618666249 podStartE2EDuration="4.354244444s" totalImagesPullingTime="2.735578195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:15:13.352035307 +0000 UTC m=+643.574563543" watchObservedRunningTime="2026-07-01 23:15:13.354244444 +0000 UTC m=+643.576772654" Jul 01 23:15:21.359791 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:21.359758 2569 generic.go:356] "Generic (PLEG): container finished" podID="0283a6bf-b13e-4dd3-a602-61cf1d715db8" containerID="121eb763875dfe85dad69fa02e9a8bc35ddd86096954cbdbf5285df9d266236b" exitCode=0 Jul 01 23:15:21.360200 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:21.359832 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" event={"ID":"0283a6bf-b13e-4dd3-a602-61cf1d715db8","Type":"ContainerDied","Data":"121eb763875dfe85dad69fa02e9a8bc35ddd86096954cbdbf5285df9d266236b"} Jul 01 23:15:21.360288 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:21.360268 2569 scope.go:117] "RemoveContainer" containerID="121eb763875dfe85dad69fa02e9a8bc35ddd86096954cbdbf5285df9d266236b" Jul 01 23:15:21.361247 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:21.361221 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:15:21.361312 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:21.361288 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:15:21.361352 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:21.361319 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 23:15:22.367326 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:22.367295 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" event={"ID":"0283a6bf-b13e-4dd3-a602-61cf1d715db8","Type":"ContainerStarted","Data":"51916d641a24c5e3bb29ad70b6da5fee0b66ec3877b5cca818d3d90b92e5798f"} Jul 01 23:15:22.553365 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:22.553339 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:15:23.137863 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:23.137824 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:15:23.139090 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:23.139063 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b5l2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 01 23:15:23.139206 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:23.139115 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" podUID="0283a6bf-b13e-4dd3-a602-61cf1d715db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 01 23:15:24.137454 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.137416 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b5l2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 01 23:15:24.137882 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.137474 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" podUID="0283a6bf-b13e-4dd3-a602-61cf1d715db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 01 23:15:24.323488 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.323455 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:24.323678 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.323534 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77d44c68c8-4nv79" Jul 01 23:15:24.403354 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.403258 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f89f798cf-vcxzg"] Jul 01 23:15:24.405114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.405078 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" podUID="0f59fc1c-db76-4155-84fc-d348c5c8bddf" containerName="manager" containerID="cri-o://d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa" gracePeriod=30 Jul 01 23:15:24.693863 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.693787 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:24.764545 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.764516 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert\") pod \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " Jul 01 23:15:24.764732 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.764550 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/0f59fc1c-db76-4155-84fc-d348c5c8bddf-kube-api-access-ld7hm\") pod \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\" (UID: \"0f59fc1c-db76-4155-84fc-d348c5c8bddf\") " Jul 01 23:15:24.786812 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.785975 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f59fc1c-db76-4155-84fc-d348c5c8bddf-kube-api-access-ld7hm" (OuterVolumeSpecName: "kube-api-access-ld7hm") pod "0f59fc1c-db76-4155-84fc-d348c5c8bddf" (UID: "0f59fc1c-db76-4155-84fc-d348c5c8bddf"). InnerVolumeSpecName "kube-api-access-ld7hm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:15:24.790885 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.790858 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0f59fc1c-db76-4155-84fc-d348c5c8bddf" (UID: "0f59fc1c-db76-4155-84fc-d348c5c8bddf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:15:24.865322 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.865293 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f59fc1c-db76-4155-84fc-d348c5c8bddf-webhook-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:15:24.865322 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:24.865318 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld7hm\" (UniqueName: \"kubernetes.io/projected/0f59fc1c-db76-4155-84fc-d348c5c8bddf-kube-api-access-ld7hm\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:15:25.137888 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.137852 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b5l2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 01 23:15:25.138262 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.137909 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" podUID="0283a6bf-b13e-4dd3-a602-61cf1d715db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 01 23:15:25.418513 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.418411 2569 generic.go:356] "Generic (PLEG): container finished" podID="0f59fc1c-db76-4155-84fc-d348c5c8bddf" containerID="d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa" exitCode=0 Jul 01 23:15:25.418513 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.418479 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" Jul 01 23:15:25.418800 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.418510 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" event={"ID":"0f59fc1c-db76-4155-84fc-d348c5c8bddf","Type":"ContainerDied","Data":"d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa"} Jul 01 23:15:25.418800 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.418601 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f89f798cf-vcxzg" event={"ID":"0f59fc1c-db76-4155-84fc-d348c5c8bddf","Type":"ContainerDied","Data":"34e57aaaf938a5b8c93f0f8e9efd1814851e6a1e9788ba131a6080337a6b9bc0"} Jul 01 23:15:25.418800 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.418655 2569 scope.go:117] "RemoveContainer" containerID="d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa" Jul 01 23:15:25.428075 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.428061 2569 scope.go:117] "RemoveContainer" containerID="d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa" Jul 01 23:15:25.428327 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:15:25.428309 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa\": container with ID starting with d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa not found: ID does not exist" containerID="d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa" Jul 01 23:15:25.428374 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.428334 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa"} err="failed to get container status \"d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa\": rpc error: code = NotFound desc = could not find container \"d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa\": container with ID starting with d7e2018257b63509be08ea4e0f00563534cb91f4298d9c547ae462a1e8e99caa not found: ID does not exist" Jul 01 23:15:25.439000 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.438978 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f89f798cf-vcxzg"] Jul 01 23:15:25.448614 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:25.448586 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f89f798cf-vcxzg"] Jul 01 23:15:26.137391 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:26.137359 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b5l2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 01 23:15:26.137565 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:26.137420 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" podUID="0283a6bf-b13e-4dd3-a602-61cf1d715db8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 01 23:15:26.923401 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:26.923367 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f59fc1c-db76-4155-84fc-d348c5c8bddf" path="/var/lib/kubelet/pods/0f59fc1c-db76-4155-84fc-d348c5c8bddf/volumes" Jul 01 23:15:27.141224 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:27.141196 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:15:27.142356 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:27.142335 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b5l2x" Jul 01 23:15:44.923583 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.923551 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-69f5665455-5bf67"] Jul 01 23:15:44.924101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.923965 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f59fc1c-db76-4155-84fc-d348c5c8bddf" containerName="manager" Jul 01 23:15:44.924101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.923977 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f59fc1c-db76-4155-84fc-d348c5c8bddf" containerName="manager" Jul 01 23:15:44.924101 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.924049 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f59fc1c-db76-4155-84fc-d348c5c8bddf" containerName="manager" Jul 01 23:15:44.929543 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.929523 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:44.931994 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.931971 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 01 23:15:44.932118 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.932037 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-ldx6c\"" Jul 01 23:15:44.937646 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:44.937623 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-69f5665455-5bf67"] Jul 01 23:15:45.047363 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.047328 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-openshift-service-ca\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.047549 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.047386 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxq9n\" (UniqueName: \"kubernetes.io/projected/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-kube-api-access-vxq9n\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.047549 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.047461 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-maas-api-tls\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.147912 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.147877 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-maas-api-tls\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.148076 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.147926 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-openshift-service-ca\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.148076 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.147981 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxq9n\" (UniqueName: \"kubernetes.io/projected/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-kube-api-access-vxq9n\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.148688 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.148671 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-openshift-service-ca\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.150319 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.150295 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-maas-api-tls\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.159369 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.159349 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxq9n\" (UniqueName: \"kubernetes.io/projected/bd29b2e0-146c-4e5e-aa36-5efa13fdc80f-kube-api-access-vxq9n\") pod \"maas-api-69f5665455-5bf67\" (UID: \"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f\") " pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.241682 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.241588 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:45.407572 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.407548 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-69f5665455-5bf67"] Jul 01 23:15:45.408002 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:15:45.407972 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd29b2e0_146c_4e5e_aa36_5efa13fdc80f.slice/crio-6c2abc2a9afb55575d6d581c2514734c76d417a7a5d01b8978fd738135def661 WatchSource:0}: Error finding container 6c2abc2a9afb55575d6d581c2514734c76d417a7a5d01b8978fd738135def661: Status 404 returned error can't find the container with id 6c2abc2a9afb55575d6d581c2514734c76d417a7a5d01b8978fd738135def661 Jul 01 23:15:45.533981 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.533900 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-69f5665455-5bf67" event={"ID":"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f","Type":"ContainerStarted","Data":"6c2abc2a9afb55575d6d581c2514734c76d417a7a5d01b8978fd738135def661"} Jul 01 23:15:45.824850 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:45.824815 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:15:47.546577 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:47.546535 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-69f5665455-5bf67" event={"ID":"bd29b2e0-146c-4e5e-aa36-5efa13fdc80f","Type":"ContainerStarted","Data":"d82a9b5ab1689141889b18a4a8e87099996f354df9913ae5ad5fa61ed12865ce"} Jul 01 23:15:47.547007 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:47.546721 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:15:47.565182 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:47.565132 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-69f5665455-5bf67" podStartSLOduration=1.9700562659999998 podStartE2EDuration="3.565117597s" totalImagesPullingTime="1.595061331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:15:47.564523966 +0000 UTC m=+677.787052182" watchObservedRunningTime="2026-07-01 23:15:47.565117597 +0000 UTC m=+677.787645808" Jul 01 23:15:53.557428 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:15:53.557402 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-69f5665455-5bf67" Jul 01 23:16:34.348447 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.348370 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74cdb96c85-q47mp"] Jul 01 23:16:34.352011 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.351996 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.355235 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.355213 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t64dd\"" Jul 01 23:16:34.355364 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.355215 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jul 01 23:16:34.355364 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.355215 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 01 23:16:34.367644 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.367621 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74cdb96c85-q47mp"] Jul 01 23:16:34.394122 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.394103 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae73251c-522a-4da7-82f0-6ef953ee8ca6-tls-cert\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.394229 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.394148 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8sb\" (UniqueName: \"kubernetes.io/projected/ae73251c-522a-4da7-82f0-6ef953ee8ca6-kube-api-access-xp8sb\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.394272 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.394245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ae73251c-522a-4da7-82f0-6ef953ee8ca6-oidc-ca\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.494874 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.494847 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ae73251c-522a-4da7-82f0-6ef953ee8ca6-oidc-ca\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.494997 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.494900 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae73251c-522a-4da7-82f0-6ef953ee8ca6-tls-cert\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.494997 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.494959 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8sb\" (UniqueName: \"kubernetes.io/projected/ae73251c-522a-4da7-82f0-6ef953ee8ca6-kube-api-access-xp8sb\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.495621 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.495579 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ae73251c-522a-4da7-82f0-6ef953ee8ca6-oidc-ca\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.497359 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.497337 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae73251c-522a-4da7-82f0-6ef953ee8ca6-tls-cert\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.505453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.505432 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8sb\" (UniqueName: \"kubernetes.io/projected/ae73251c-522a-4da7-82f0-6ef953ee8ca6-kube-api-access-xp8sb\") pod \"authorino-74cdb96c85-q47mp\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.661698 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.661589 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:16:34.800719 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.800639 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74cdb96c85-q47mp"] Jul 01 23:16:34.801800 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:16:34.801768 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae73251c_522a_4da7_82f0_6ef953ee8ca6.slice/crio-607fee9340bbdb269fc0d1d2a417bea93bac597ab2c06cd6da88bd9dd8a80c0a WatchSource:0}: Error finding container 607fee9340bbdb269fc0d1d2a417bea93bac597ab2c06cd6da88bd9dd8a80c0a: Status 404 returned error can't find the container with id 607fee9340bbdb269fc0d1d2a417bea93bac597ab2c06cd6da88bd9dd8a80c0a Jul 01 23:16:34.820223 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:34.820198 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cdb96c85-q47mp" event={"ID":"ae73251c-522a-4da7-82f0-6ef953ee8ca6","Type":"ContainerStarted","Data":"607fee9340bbdb269fc0d1d2a417bea93bac597ab2c06cd6da88bd9dd8a80c0a"} Jul 01 23:16:35.826947 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:35.826906 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cdb96c85-q47mp" event={"ID":"ae73251c-522a-4da7-82f0-6ef953ee8ca6","Type":"ContainerStarted","Data":"043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c"} Jul 01 23:16:52.950225 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:52.950172 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74cdb96c85-q47mp" podStartSLOduration=18.598936034 podStartE2EDuration="18.950153404s" totalImagesPullingTime="351.21737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:16:35.844632215 +0000 UTC m=+726.067160426" watchObservedRunningTime="2026-07-01 23:16:52.950153404 +0000 UTC m=+743.172681614" Jul 01 23:16:52.955625 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:52.953759 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:16:58.589099 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:16:58.589062 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:17:08.953963 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.953925 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc"] Jul 01 23:17:08.957679 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.957646 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:08.961725 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.961528 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 01 23:17:08.961725 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.961713 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-czjtt\"" Jul 01 23:17:08.961923 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.961750 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 01 23:17:08.963756 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.963734 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 01 23:17:08.975422 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:08.975404 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc"] Jul 01 23:17:09.020683 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.020655 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.020820 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.020696 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vv5r\" (UniqueName: \"kubernetes.io/projected/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-kube-api-access-2vv5r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.020820 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.020769 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.020820 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.020814 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.020922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.020854 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.020922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.020907 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.121840 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.121811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vv5r\" (UniqueName: \"kubernetes.io/projected/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-kube-api-access-2vv5r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122034 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.121851 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122034 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.121878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122157 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.122041 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122157 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.122088 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122157 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.122125 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122306 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.122206 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122306 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.122294 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.122414 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.122393 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.124111 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.124074 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.124350 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.124334 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.130499 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.130480 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vv5r\" (UniqueName: \"kubernetes.io/projected/a0bb4851-0cd4-46e4-a24d-3f1017dc3b76-kube-api-access-2vv5r\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qfdhc\" (UID: \"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.270532 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.270449 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:09.417873 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:09.417839 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc"] Jul 01 23:17:09.419070 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:17:09.419042 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0bb4851_0cd4_46e4_a24d_3f1017dc3b76.slice/crio-d94209b7abbf2c4e01af8b9779383bb36ff1cff8d1b47f97906de3a51436b406 WatchSource:0}: Error finding container d94209b7abbf2c4e01af8b9779383bb36ff1cff8d1b47f97906de3a51436b406: Status 404 returned error can't find the container with id d94209b7abbf2c4e01af8b9779383bb36ff1cff8d1b47f97906de3a51436b406 Jul 01 23:17:10.042219 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:10.042183 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" event={"ID":"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76","Type":"ContainerStarted","Data":"d94209b7abbf2c4e01af8b9779383bb36ff1cff8d1b47f97906de3a51436b406"} Jul 01 23:17:10.448399 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:10.448365 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:17:11.050458 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:11.050415 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" event={"ID":"a0bb4851-0cd4-46e4-a24d-3f1017dc3b76","Type":"ContainerStarted","Data":"7917ecb16f0e2f79cdc868e4024cc8bb91ec452fbbfaa612471753660e2fa62f"} Jul 01 23:17:11.074486 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:11.074397 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" podStartSLOduration=2.036786914 podStartE2EDuration="3.074374208s" totalImagesPullingTime="1.037587294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:17:11.07129874 +0000 UTC m=+761.293826987" watchObservedRunningTime="2026-07-01 23:17:11.074374208 +0000 UTC m=+761.296902421" Jul 01 23:17:19.270668 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:19.270631 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:19.271150 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:19.270710 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:19.283170 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:19.283150 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:20.116230 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:20.116203 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qfdhc" Jul 01 23:17:25.523703 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:25.523663 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:17:33.838074 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:33.838035 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:17:38.432637 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.432581 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl"] Jul 01 23:17:38.436352 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.436336 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.438711 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.438692 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 01 23:17:38.445784 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.445757 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl"] Jul 01 23:17:38.596223 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.596191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.596394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.596243 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwpx\" (UniqueName: \"kubernetes.io/projected/3868714c-75b1-4609-a764-4fa7461912f4-kube-api-access-8qwpx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.596394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.596271 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.596394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.596307 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.596394 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.596369 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.596577 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.596402 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3868714c-75b1-4609-a764-4fa7461912f4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697340 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697340 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwpx\" (UniqueName: \"kubernetes.io/projected/3868714c-75b1-4609-a764-4fa7461912f4-kube-api-access-8qwpx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697340 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697324 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697671 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697352 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697671 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697381 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697671 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3868714c-75b1-4609-a764-4fa7461912f4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697870 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697837 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697866 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.697922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.697895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.699687 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.699657 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3868714c-75b1-4609-a764-4fa7461912f4-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.699813 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.699797 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3868714c-75b1-4609-a764-4fa7461912f4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.705283 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.705265 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwpx\" (UniqueName: \"kubernetes.io/projected/3868714c-75b1-4609-a764-4fa7461912f4-kube-api-access-8qwpx\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl\" (UID: \"3868714c-75b1-4609-a764-4fa7461912f4\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.746511 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.746472 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:38.884027 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:38.884002 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl"] Jul 01 23:17:38.885906 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:17:38.885873 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3868714c_75b1_4609_a764_4fa7461912f4.slice/crio-658622f81196a01aaee48424b05cee088107d380e0dade29f902f4fa04569d81 WatchSource:0}: Error finding container 658622f81196a01aaee48424b05cee088107d380e0dade29f902f4fa04569d81: Status 404 returned error can't find the container with id 658622f81196a01aaee48424b05cee088107d380e0dade29f902f4fa04569d81 Jul 01 23:17:39.220633 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:39.220590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" event={"ID":"3868714c-75b1-4609-a764-4fa7461912f4","Type":"ContainerStarted","Data":"70017fdfff4287028e6f494f77af0a9759f1f8a57a461fcca8872b8c7368ac0a"} Jul 01 23:17:39.220633 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:39.220642 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" event={"ID":"3868714c-75b1-4609-a764-4fa7461912f4","Type":"ContainerStarted","Data":"658622f81196a01aaee48424b05cee088107d380e0dade29f902f4fa04569d81"} Jul 01 23:17:39.242641 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:39.242577 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" podStartSLOduration=1.082375457 podStartE2EDuration="1.242561811s" totalImagesPullingTime="160.186354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:17:39.240860929 +0000 UTC m=+789.463389140" watchObservedRunningTime="2026-07-01 23:17:39.242561811 +0000 UTC m=+789.465090022" Jul 01 23:17:41.907770 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:41.902485 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gc8l6"] Jul 01 23:17:48.746995 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:48.746958 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:48.747469 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:48.747055 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:48.760136 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:48.760115 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:17:49.289369 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:17:49.289339 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl" Jul 01 23:18:47.817890 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:47.817854 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74cdb96c85-q47mp"] Jul 01 23:18:47.818433 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:47.818041 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-74cdb96c85-q47mp" podUID="ae73251c-522a-4da7-82f0-6ef953ee8ca6" containerName="authorino" containerID="cri-o://043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c" gracePeriod=30 Jul 01 23:18:48.077085 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.077031 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:18:48.162038 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.161996 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae73251c-522a-4da7-82f0-6ef953ee8ca6-tls-cert\") pod \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " Jul 01 23:18:48.162410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.162392 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ae73251c-522a-4da7-82f0-6ef953ee8ca6-oidc-ca\") pod \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " Jul 01 23:18:48.162627 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.162592 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp8sb\" (UniqueName: \"kubernetes.io/projected/ae73251c-522a-4da7-82f0-6ef953ee8ca6-kube-api-access-xp8sb\") pod \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\" (UID: \"ae73251c-522a-4da7-82f0-6ef953ee8ca6\") " Jul 01 23:18:48.165088 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.165055 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae73251c-522a-4da7-82f0-6ef953ee8ca6-kube-api-access-xp8sb" (OuterVolumeSpecName: "kube-api-access-xp8sb") pod "ae73251c-522a-4da7-82f0-6ef953ee8ca6" (UID: "ae73251c-522a-4da7-82f0-6ef953ee8ca6"). InnerVolumeSpecName "kube-api-access-xp8sb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 23:18:48.170187 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.170154 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae73251c-522a-4da7-82f0-6ef953ee8ca6-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "ae73251c-522a-4da7-82f0-6ef953ee8ca6" (UID: "ae73251c-522a-4da7-82f0-6ef953ee8ca6"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 23:18:48.177335 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.177307 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae73251c-522a-4da7-82f0-6ef953ee8ca6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ae73251c-522a-4da7-82f0-6ef953ee8ca6" (UID: "ae73251c-522a-4da7-82f0-6ef953ee8ca6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 23:18:48.264723 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.264693 2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae73251c-522a-4da7-82f0-6ef953ee8ca6-tls-cert\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:18:48.264723 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.264719 2569 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ae73251c-522a-4da7-82f0-6ef953ee8ca6-oidc-ca\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:18:48.264922 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.264731 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp8sb\" (UniqueName: \"kubernetes.io/projected/ae73251c-522a-4da7-82f0-6ef953ee8ca6-kube-api-access-xp8sb\") on node \"ip-10-0-141-234.ec2.internal\" DevicePath \"\"" Jul 01 23:18:48.635420 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.635382 2569 generic.go:356] "Generic (PLEG): container finished" podID="ae73251c-522a-4da7-82f0-6ef953ee8ca6" containerID="043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c" exitCode=0 Jul 01 23:18:48.635648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.635445 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74cdb96c85-q47mp" Jul 01 23:18:48.635648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.635478 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cdb96c85-q47mp" event={"ID":"ae73251c-522a-4da7-82f0-6ef953ee8ca6","Type":"ContainerDied","Data":"043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c"} Jul 01 23:18:48.635648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.635526 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74cdb96c85-q47mp" event={"ID":"ae73251c-522a-4da7-82f0-6ef953ee8ca6","Type":"ContainerDied","Data":"607fee9340bbdb269fc0d1d2a417bea93bac597ab2c06cd6da88bd9dd8a80c0a"} Jul 01 23:18:48.635648 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.635544 2569 scope.go:117] "RemoveContainer" containerID="043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c" Jul 01 23:18:48.645456 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.645439 2569 scope.go:117] "RemoveContainer" containerID="043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c" Jul 01 23:18:48.645745 ip-10-0-141-234 kubenswrapper[2569]: E0701 23:18:48.645727 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c\": container with ID starting with 043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c not found: ID does not exist" containerID="043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c" Jul 01 23:18:48.645805 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.645754 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c"} err="failed to get container status \"043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c\": rpc error: code = NotFound desc = could not find container \"043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c\": container with ID starting with 043622f948933e2a6a2643408632a5a6fd8e4bcda6bcffaf421bf7d159f4408c not found: ID does not exist" Jul 01 23:18:48.656730 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.656709 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74cdb96c85-q47mp"] Jul 01 23:18:48.660788 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.660768 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74cdb96c85-q47mp"] Jul 01 23:18:48.923722 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:18:48.923645 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae73251c-522a-4da7-82f0-6ef953ee8ca6" path="/var/lib/kubelet/pods/ae73251c-522a-4da7-82f0-6ef953ee8ca6/volumes" Jul 01 23:19:30.337058 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:30.336984 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:19:30.338926 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:30.338900 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:19:30.937383 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:30.937345 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gn76v_0416fbc7-e700-4864-aa64-10b524e14ff9/manager/0.log" Jul 01 23:19:31.058694 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.058659 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rfhb5_152fc74c-3a9c-40ec-ba5a-9d94c7848149/manager/0.log" Jul 01 23:19:31.167950 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.167919 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-69f5665455-5bf67_bd29b2e0-146c-4e5e-aa36-5efa13fdc80f/maas-api/0.log" Jul 01 23:19:31.286650 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.286551 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77d44c68c8-4nv79_d49e6287-a7c5-41f1-950e-f55dbaa31190/manager/0.log" Jul 01 23:19:31.399333 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.399308 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6rb82_19d5c636-734f-490d-a0c1-348c9727dff6/server/0.log" Jul 01 23:19:31.528004 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.527971 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mq9fx_0cde6afd-ff4d-4720-a076-e937faf27b99/manager/0.log" Jul 01 23:19:31.789772 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.789734 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj_266a292b-1ade-4700-93c3-fcedebf0b378/manager/0.log" Jul 01 23:19:31.798621 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:31.798582 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj_266a292b-1ade-4700-93c3-fcedebf0b378/copy-manifests/0.log" Jul 01 23:19:32.786006 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:32.785975 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn_9df7591f-47f8-4f6e-a891-23dd5f0d156b/util/0.log" Jul 01 23:19:32.793566 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:32.793539 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn_9df7591f-47f8-4f6e-a891-23dd5f0d156b/pull/0.log" Jul 01 23:19:32.800167 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:32.800147 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn_9df7591f-47f8-4f6e-a891-23dd5f0d156b/extract/0.log" Jul 01 23:19:32.912144 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:32.912112 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6_0255ff73-fd18-4f66-ba54-b96c83208dab/extract/0.log" Jul 01 23:19:32.918159 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:32.918138 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6_0255ff73-fd18-4f66-ba54-b96c83208dab/util/0.log" Jul 01 23:19:32.926236 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:32.926205 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6_0255ff73-fd18-4f66-ba54-b96c83208dab/pull/0.log" Jul 01 23:19:33.034550 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.034524 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk_dad158f4-72d9-438f-9615-98445a2019e0/util/0.log" Jul 01 23:19:33.041011 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.040948 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk_dad158f4-72d9-438f-9615-98445a2019e0/pull/0.log" Jul 01 23:19:33.047179 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.047160 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk_dad158f4-72d9-438f-9615-98445a2019e0/extract/0.log" Jul 01 23:19:33.159155 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.159134 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn_644a2067-c25b-416a-83b3-a543f2842142/util/0.log" Jul 01 23:19:33.165453 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.165433 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn_644a2067-c25b-416a-83b3-a543f2842142/pull/0.log" Jul 01 23:19:33.171988 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.171972 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn_644a2067-c25b-416a-83b3-a543f2842142/extract/0.log" Jul 01 23:19:33.407073 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.407043 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jdfvf_7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9/manager/0.log" Jul 01 23:19:33.515025 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.514996 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qb9ht_98a2becf-f5ff-46d8-8d67-a3030a5ae191/manager/0.log" Jul 01 23:19:33.742343 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.742255 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4cmxt_81fd390d-b0de-4352-9de5-62d14e80b0a4/registry-server/0.log" Jul 01 23:19:33.869484 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.869459 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-42mrj_9a980a54-77ac-4443-93df-0679af367725/manager/0.log" Jul 01 23:19:33.997260 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:33.997182 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gc8l6_fa57adf9-9295-432c-871b-f1dd0313d4ca/limitador/0.log" Jul 01 23:19:34.112933 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:34.112904 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-f4b8q_3fe29758-690b-4ea7-83af-b2c70b9db594/manager/0.log" Jul 01 23:19:34.786239 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:34.786211 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869858444d-xzcrk_a9cd41c9-e389-4786-9656-54be4100f511/kube-auth-proxy/0.log" Jul 01 23:19:34.900921 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:34.900893 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/1.log" Jul 01 23:19:35.786711 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:35.786678 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qfdhc_a0bb4851-0cd4-46e4-a24d-3f1017dc3b76/main/0.log" Jul 01 23:19:36.136577 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:36.136546 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-h56sl_3868714c-75b1-4609-a764-4fa7461912f4/main/0.log" Jul 01 23:19:40.353972 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.353938 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6fqcv/must-gather-p669w"] Jul 01 23:19:40.354421 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.354407 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae73251c-522a-4da7-82f0-6ef953ee8ca6" containerName="authorino" Jul 01 23:19:40.354466 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.354424 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae73251c-522a-4da7-82f0-6ef953ee8ca6" containerName="authorino" Jul 01 23:19:40.354526 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.354516 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae73251c-522a-4da7-82f0-6ef953ee8ca6" containerName="authorino" Jul 01 23:19:40.359184 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.359169 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.361928 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.361905 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6fqcv\"/\"kube-root-ca.crt\"" Jul 01 23:19:40.363121 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.363094 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6fqcv\"/\"openshift-service-ca.crt\"" Jul 01 23:19:40.363230 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.363166 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6fqcv\"/\"default-dockercfg-k55ns\"" Jul 01 23:19:40.366481 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.366458 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6fqcv/must-gather-p669w"] Jul 01 23:19:40.533178 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.533141 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c794\" (UniqueName: \"kubernetes.io/projected/ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5-kube-api-access-2c794\") pod \"must-gather-p669w\" (UID: \"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5\") " pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.533359 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.533210 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5-must-gather-output\") pod \"must-gather-p669w\" (UID: \"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5\") " pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.634031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.633939 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c794\" (UniqueName: \"kubernetes.io/projected/ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5-kube-api-access-2c794\") pod \"must-gather-p669w\" (UID: \"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5\") " pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.634031 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.634002 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5-must-gather-output\") pod \"must-gather-p669w\" (UID: \"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5\") " pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.634345 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.634325 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5-must-gather-output\") pod \"must-gather-p669w\" (UID: \"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5\") " pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.641985 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.641956 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c794\" (UniqueName: \"kubernetes.io/projected/ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5-kube-api-access-2c794\") pod \"must-gather-p669w\" (UID: \"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5\") " pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.670162 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.670131 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6fqcv/must-gather-p669w" Jul 01 23:19:40.810359 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.810335 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6fqcv/must-gather-p669w"] Jul 01 23:19:40.812152 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:19:40.812120 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae1b882e_dba2_4efc_a0d3_a6fadfa6d2b5.slice/crio-1b7eb7acdf0691f497454a5422baae9df8913f8886ed7a4db2e4b1c38de249d7 WatchSource:0}: Error finding container 1b7eb7acdf0691f497454a5422baae9df8913f8886ed7a4db2e4b1c38de249d7: Status 404 returned error can't find the container with id 1b7eb7acdf0691f497454a5422baae9df8913f8886ed7a4db2e4b1c38de249d7 Jul 01 23:19:40.939404 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:40.939316 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fqcv/must-gather-p669w" event={"ID":"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5","Type":"ContainerStarted","Data":"1b7eb7acdf0691f497454a5422baae9df8913f8886ed7a4db2e4b1c38de249d7"} Jul 01 23:19:41.948851 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:41.948556 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fqcv/must-gather-p669w" event={"ID":"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5","Type":"ContainerStarted","Data":"dfb0220e41b381d87f50fcb5785d014403e8630e86e274ac0dc11c18dac4433c"} Jul 01 23:19:41.948851 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:41.948626 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fqcv/must-gather-p669w" event={"ID":"ae1b882e-dba2-4efc-a0d3-a6fadfa6d2b5","Type":"ContainerStarted","Data":"46c93dcf4dd3a2cc7855dedc3b74fc162e0b0e36456174711d00209c202c4ea6"} Jul 01 23:19:41.968659 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:41.968580 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6fqcv/must-gather-p669w" podStartSLOduration=1.054051528 podStartE2EDuration="1.968558812s" totalImagesPullingTime="914.507284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 23:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:19:41.966750115 +0000 UTC m=+912.189278326" watchObservedRunningTime="2026-07-01 23:19:41.968558812 +0000 UTC m=+912.191087025" Jul 01 23:19:43.400256 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:43.400216 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xjqgl_afd99605-6853-41d1-8b9f-a4aa46695472/global-pull-secret-syncer/0.log" Jul 01 23:19:43.515307 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:43.515250 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tz5vp_d6d6e1ef-fd1d-4a47-a28c-16b1343110b9/konnectivity-agent/0.log" Jul 01 23:19:43.581511 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:43.581482 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-234.ec2.internal_cec0e47b4d0f2d635d3199e833873acd/haproxy/0.log" Jul 01 23:19:47.310014 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.309981 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn_9df7591f-47f8-4f6e-a891-23dd5f0d156b/extract/0.log" Jul 01 23:19:47.331181 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.331149 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn_9df7591f-47f8-4f6e-a891-23dd5f0d156b/util/0.log" Jul 01 23:19:47.355116 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.355082 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rsprn_9df7591f-47f8-4f6e-a891-23dd5f0d156b/pull/0.log" Jul 01 23:19:47.398244 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.398214 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6_0255ff73-fd18-4f66-ba54-b96c83208dab/extract/0.log" Jul 01 23:19:47.414171 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.414140 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6_0255ff73-fd18-4f66-ba54-b96c83208dab/util/0.log" Jul 01 23:19:47.448004 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.447979 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ptdf6_0255ff73-fd18-4f66-ba54-b96c83208dab/pull/0.log" Jul 01 23:19:47.482073 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.482040 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk_dad158f4-72d9-438f-9615-98445a2019e0/extract/0.log" Jul 01 23:19:47.503638 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.503589 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk_dad158f4-72d9-438f-9615-98445a2019e0/util/0.log" Jul 01 23:19:47.527148 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.527090 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tlltk_dad158f4-72d9-438f-9615-98445a2019e0/pull/0.log" Jul 01 23:19:47.555571 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.555533 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn_644a2067-c25b-416a-83b3-a543f2842142/extract/0.log" Jul 01 23:19:47.583999 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.583966 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn_644a2067-c25b-416a-83b3-a543f2842142/util/0.log" Jul 01 23:19:47.615627 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.615468 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lrxgn_644a2067-c25b-416a-83b3-a543f2842142/pull/0.log" Jul 01 23:19:47.697419 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.697384 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jdfvf_7aa7f8e1-9aac-48a6-82d4-361d4ed50fc9/manager/0.log" Jul 01 23:19:47.737407 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.737366 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qb9ht_98a2becf-f5ff-46d8-8d67-a3030a5ae191/manager/0.log" Jul 01 23:19:47.795626 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.792139 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4cmxt_81fd390d-b0de-4352-9de5-62d14e80b0a4/registry-server/0.log" Jul 01 23:19:47.832667 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.831930 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-42mrj_9a980a54-77ac-4443-93df-0679af367725/manager/0.log" Jul 01 23:19:47.865022 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.864947 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gc8l6_fa57adf9-9295-432c-871b-f1dd0313d4ca/limitador/0.log" Jul 01 23:19:47.899410 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:47.899369 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-f4b8q_3fe29758-690b-4ea7-83af-b2c70b9db594/manager/0.log" Jul 01 23:19:49.597308 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.597275 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/alertmanager/0.log" Jul 01 23:19:49.624520 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.624493 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/config-reloader/0.log" Jul 01 23:19:49.646687 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.646656 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/kube-rbac-proxy-web/0.log" Jul 01 23:19:49.670344 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.670315 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/kube-rbac-proxy/0.log" Jul 01 23:19:49.695048 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.695017 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/kube-rbac-proxy-metric/0.log" Jul 01 23:19:49.716796 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.716768 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/prom-label-proxy/0.log" Jul 01 23:19:49.741540 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.741509 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cdc4d633-9b66-49db-beb6-282be6f36a35/init-config-reloader/0.log" Jul 01 23:19:49.805668 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.805637 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-8c6fh_8884db4b-140d-4ee0-ba84-4ae339700754/kube-state-metrics/0.log" Jul 01 23:19:49.824941 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.824909 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-8c6fh_8884db4b-140d-4ee0-ba84-4ae339700754/kube-rbac-proxy-main/0.log" Jul 01 23:19:49.854560 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.854480 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-8c6fh_8884db4b-140d-4ee0-ba84-4ae339700754/kube-rbac-proxy-self/0.log" Jul 01 23:19:49.931091 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:49.931059 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-pnc95_abc751df-5651-45e3-97b6-b1107ab2dc6c/monitoring-plugin/0.log" Jul 01 23:19:50.129512 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.129428 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsxv9_3fbee254-bf29-46fa-b44a-a104ee5e9ab2/node-exporter/0.log" Jul 01 23:19:50.148655 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.148552 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsxv9_3fbee254-bf29-46fa-b44a-a104ee5e9ab2/kube-rbac-proxy/0.log" Jul 01 23:19:50.169660 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.169629 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wsxv9_3fbee254-bf29-46fa-b44a-a104ee5e9ab2/init-textfile/0.log" Jul 01 23:19:50.199572 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.199542 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-g2rqv_3ecf1b55-0754-4c71-84b7-36b67f3d077c/kube-rbac-proxy-main/0.log" Jul 01 23:19:50.223082 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.223050 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-g2rqv_3ecf1b55-0754-4c71-84b7-36b67f3d077c/kube-rbac-proxy-self/0.log" Jul 01 23:19:50.254381 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.254349 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-g2rqv_3ecf1b55-0754-4c71-84b7-36b67f3d077c/openshift-state-metrics/0.log" Jul 01 23:19:50.526627 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.526466 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-hc8rs_236ebef7-24b0-45ad-8b90-edf250868228/prometheus-operator/0.log" Jul 01 23:19:50.559686 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.559562 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-hc8rs_236ebef7-24b0-45ad-8b90-edf250868228/kube-rbac-proxy/0.log" Jul 01 23:19:50.616947 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.616869 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-89rvs_c8711075-504f-47f5-969c-45b236f501e2/prometheus-operator-admission-webhook/0.log" Jul 01 23:19:50.649047 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.648343 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74bcbbdbf8-pv8hk_b4287f19-1583-44c5-aa41-380e32238c0e/telemeter-client/0.log" Jul 01 23:19:50.670578 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.670543 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74bcbbdbf8-pv8hk_b4287f19-1583-44c5-aa41-380e32238c0e/reload/0.log" Jul 01 23:19:50.693789 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.693758 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74bcbbdbf8-pv8hk_b4287f19-1583-44c5-aa41-380e32238c0e/kube-rbac-proxy/0.log" Jul 01 23:19:50.728479 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.728401 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9b5b7cd-jfg5z_29399438-37e1-46f3-b3d6-563eb4d118d7/thanos-query/0.log" Jul 01 23:19:50.750601 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.750559 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9b5b7cd-jfg5z_29399438-37e1-46f3-b3d6-563eb4d118d7/kube-rbac-proxy-web/0.log" Jul 01 23:19:50.781494 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.781455 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9b5b7cd-jfg5z_29399438-37e1-46f3-b3d6-563eb4d118d7/kube-rbac-proxy/0.log" Jul 01 23:19:50.808111 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.807938 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9b5b7cd-jfg5z_29399438-37e1-46f3-b3d6-563eb4d118d7/prom-label-proxy/0.log" Jul 01 23:19:50.837668 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.837634 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9b5b7cd-jfg5z_29399438-37e1-46f3-b3d6-563eb4d118d7/kube-rbac-proxy-rules/0.log" Jul 01 23:19:50.863174 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:50.863145 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9b5b7cd-jfg5z_29399438-37e1-46f3-b3d6-563eb4d118d7/kube-rbac-proxy-metrics/0.log" Jul 01 23:19:53.445445 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.445412 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77dd554656-mbmf2_019f2a75-0405-4f2d-8128-d65b541ef709/console/0.log" Jul 01 23:19:53.474962 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.474921 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-vnw62_031efe8c-bb8d-489f-97fc-4195683ba594/download-server/0.log" Jul 01 23:19:53.830012 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.829975 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6"] Jul 01 23:19:53.834649 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.834590 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.843219 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.843194 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6"] Jul 01 23:19:53.886830 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.886794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc642\" (UniqueName: \"kubernetes.io/projected/cba64ae6-1724-4a5d-99aa-a7b40350f871-kube-api-access-vc642\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.887022 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.886895 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-sys\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.887022 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.886929 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-lib-modules\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.887022 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.886971 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-podres\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.887195 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.887095 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-proc\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.987915 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.987876 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc642\" (UniqueName: \"kubernetes.io/projected/cba64ae6-1724-4a5d-99aa-a7b40350f871-kube-api-access-vc642\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.987945 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-sys\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.987970 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-lib-modules\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.988009 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-podres\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988114 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.988040 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-sys\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988354 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.988169 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-podres\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988354 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.988177 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-lib-modules\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988354 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.988205 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-proc\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.988354 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.988305 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cba64ae6-1724-4a5d-99aa-a7b40350f871-proc\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:53.996832 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:53.996777 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc642\" (UniqueName: \"kubernetes.io/projected/cba64ae6-1724-4a5d-99aa-a7b40350f871-kube-api-access-vc642\") pod \"perf-node-gather-daemonset-zbjh6\" (UID: \"cba64ae6-1724-4a5d-99aa-a7b40350f871\") " pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:54.152599 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:54.152513 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:54.344484 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:54.344447 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6"] Jul 01 23:19:54.345864 ip-10-0-141-234 kubenswrapper[2569]: W0701 23:19:54.345779 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcba64ae6_1724_4a5d_99aa_a7b40350f871.slice/crio-1b64f3600d632658e4e362f5dd81a3e1a0c6d8acc2c97a7363a7a1659582de1c WatchSource:0}: Error finding container 1b64f3600d632658e4e362f5dd81a3e1a0c6d8acc2c97a7363a7a1659582de1c: Status 404 returned error can't find the container with id 1b64f3600d632658e4e362f5dd81a3e1a0c6d8acc2c97a7363a7a1659582de1c Jul 01 23:19:55.070341 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.070295 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" event={"ID":"cba64ae6-1724-4a5d-99aa-a7b40350f871","Type":"ContainerStarted","Data":"91e12de0b5b891b7859ddb6cca494cbabf630716619750f1bc354c37d18a41cd"} Jul 01 23:19:55.070830 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.070367 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" event={"ID":"cba64ae6-1724-4a5d-99aa-a7b40350f871","Type":"ContainerStarted","Data":"1b64f3600d632658e4e362f5dd81a3e1a0c6d8acc2c97a7363a7a1659582de1c"} Jul 01 23:19:55.071244 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.071214 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:19:55.091399 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.091337 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" podStartSLOduration=2.091316662 podStartE2EDuration="2.091316662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 23:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 23:19:55.089420591 +0000 UTC m=+925.311948820" watchObservedRunningTime="2026-07-01 23:19:55.091316662 +0000 UTC m=+925.313844874" Jul 01 23:19:55.337104 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.337032 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nt6sb_d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d/dns/0.log" Jul 01 23:19:55.356727 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.356690 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nt6sb_d6d1fec4-9a83-4404-aea1-b63e3eb0fe6d/kube-rbac-proxy/0.log" Jul 01 23:19:55.463698 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.463668 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vb4f8_3e982204-a919-4ae0-9ea7-1af013a9324d/dns-node-resolver/0.log" Jul 01 23:19:55.975945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:55.975916 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-ddf54f849-qx9jj_73ac79ed-a1fa-4ce2-871a-267fde34f392/registry/0.log" Jul 01 23:19:56.021257 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:56.021222 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l8tqd_b02cac67-ee6b-43a4-9151-6b30a4473f58/node-ca/0.log" Jul 01 23:19:57.107208 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:57.107126 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869858444d-xzcrk_a9cd41c9-e389-4786-9656-54be4100f511/kube-auth-proxy/0.log" Jul 01 23:19:57.139313 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:57.139282 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/0.log" Jul 01 23:19:57.139546 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:57.139394 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/1.log" Jul 01 23:19:57.809102 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:57.809078 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7hsss_07cd1d83-bd84-4673-afbc-dea22ae373e1/serve-healthcheck-canary/0.log" Jul 01 23:19:58.401376 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:58.401346 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8msw2_bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23/kube-rbac-proxy/0.log" Jul 01 23:19:58.422527 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:58.422503 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8msw2_bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23/exporter/0.log" Jul 01 23:19:58.445178 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:19:58.445146 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8msw2_bc8ea381-755d-4b7f-8f1f-b57bf5fc8d23/extractor/0.log" Jul 01 23:20:00.659564 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.659537 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gn76v_0416fbc7-e700-4864-aa64-10b524e14ff9/manager/0.log" Jul 01 23:20:00.702569 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.702541 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rfhb5_152fc74c-3a9c-40ec-ba5a-9d94c7848149/manager/0.log" Jul 01 23:20:00.726945 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.726916 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-69f5665455-5bf67_bd29b2e0-146c-4e5e-aa36-5efa13fdc80f/maas-api/0.log" Jul 01 23:20:00.766448 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.766414 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77d44c68c8-4nv79_d49e6287-a7c5-41f1-950e-f55dbaa31190/manager/0.log" Jul 01 23:20:00.786826 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.786800 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6rb82_19d5c636-734f-490d-a0c1-348c9727dff6/server/0.log" Jul 01 23:20:00.856218 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.856180 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mq9fx_0cde6afd-ff4d-4720-a076-e937faf27b99/manager/0.log" Jul 01 23:20:00.966656 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.966551 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj_266a292b-1ade-4700-93c3-fcedebf0b378/manager/0.log" Jul 01 23:20:00.982980 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:00.982946 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f4c76f6b9-fg7wj_266a292b-1ade-4700-93c3-fcedebf0b378/copy-manifests/0.log" Jul 01 23:20:02.103209 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:02.103173 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6fqcv/perf-node-gather-daemonset-zbjh6" Jul 01 23:20:03.689256 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:03.689220 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869858444d-xzcrk_a9cd41c9-e389-4786-9656-54be4100f511/kube-auth-proxy/0.log" Jul 01 23:20:03.713848 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:03.713767 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/0.log" Jul 01 23:20:03.714224 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:03.714194 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/1.log" Jul 01 23:20:05.144664 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:05.144577 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869858444d-xzcrk_a9cd41c9-e389-4786-9656-54be4100f511/kube-auth-proxy/0.log" Jul 01 23:20:05.168054 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:05.168024 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/1.log" Jul 01 23:20:05.168264 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:05.168228 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b5l2x_0283a6bf-b13e-4dd3-a602-61cf1d715db8/istio-proxy/0.log" Jul 01 23:20:09.288418 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.288395 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/kube-multus-additional-cni-plugins/0.log" Jul 01 23:20:09.310708 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.310682 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/egress-router-binary-copy/0.log" Jul 01 23:20:09.333821 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.333787 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/cni-plugins/0.log" Jul 01 23:20:09.362368 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.362335 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/bond-cni-plugin/0.log" Jul 01 23:20:09.404745 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.404720 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/routeoverride-cni/0.log" Jul 01 23:20:09.427344 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.427320 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/whereabouts-cni-bincopy/0.log" Jul 01 23:20:09.450767 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.450738 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8j64_b96d29f3-79ce-4860-92b5-3e055377dbf2/whereabouts-cni/0.log" Jul 01 23:20:09.840962 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:09.840936 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xdblm_06f9552e-9efb-4688-8ed1-10c3e64f4249/kube-multus/0.log" Jul 01 23:20:10.000828 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:10.000801 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ghqjk_f31c226b-95f7-4a3f-9b94-476f82172154/network-metrics-daemon/0.log" Jul 01 23:20:10.018779 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:10.018731 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ghqjk_f31c226b-95f7-4a3f-9b94-476f82172154/kube-rbac-proxy/0.log" Jul 01 23:20:11.315288 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.315257 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-controller/0.log" Jul 01 23:20:11.330691 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.330669 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/1.log" Jul 01 23:20:11.330816 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.330717 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovn-acl-logging/0.log" Jul 01 23:20:11.347891 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.347868 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/kube-rbac-proxy-node/0.log" Jul 01 23:20:11.368282 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.368257 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 23:20:11.388685 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.388662 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/northd/0.log" Jul 01 23:20:11.407375 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.407355 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/nbdb/0.log" Jul 01 23:20:11.426712 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.426689 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/sbdb/0.log" Jul 01 23:20:11.493557 ip-10-0-141-234 kubenswrapper[2569]: I0701 23:20:11.493524 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q5hbn_b6d8224c-48f5-425f-ab68-e7b0fe99ef4b/ovnkube-controller/0.log"