Jul 23 14:52:31.733526 ip-10-0-142-11 systemd[1]: Starting Kubernetes Kubelet... Jul 23 14:52:32.106962 ip-10-0-142-11 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 14:52:32.106962 ip-10-0-142-11 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 14:52:32.106962 ip-10-0-142-11 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 14:52:32.106962 ip-10-0-142-11 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 14:52:32.106962 ip-10-0-142-11 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 14:52:32.107962 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.107851 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 14:52:32.113712 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113690 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 14:52:32.113712 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113708 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 14:52:32.113712 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113713 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 14:52:32.113712 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113717 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113721 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113726 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113730 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113733 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113737 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113741 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113744 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113748 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113751 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113755 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113758 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113763 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113767 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113771 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113775 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113778 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113782 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113786 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113790 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 14:52:32.113956 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113794 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113798 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113802 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113806 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113810 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113814 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113817 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113821 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113825 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113830 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113834 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113840 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113844 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113850 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113855 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113859 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113863 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113867 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113871 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113875 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 14:52:32.114764 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113880 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113884 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113888 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113892 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113896 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113901 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113905 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113911 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113918 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113924 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113929 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113933 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113937 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113941 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113948 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113952 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113956 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113960 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113964 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 14:52:32.115575 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113971 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113976 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113981 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113986 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113992 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.113996 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114002 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114008 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114012 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114016 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114020 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114025 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114029 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114034 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114038 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114042 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114046 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114050 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114054 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114058 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 14:52:32.116042 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114062 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114066 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114070 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114075 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114080 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114084 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114090 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114096 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114100 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114105 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114110 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114114 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114118 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114123 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114127 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114131 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.114135 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116143 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116156 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116161 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 14:52:32.116566 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116164 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116167 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116170 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116173 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116175 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116178 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116180 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116183 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116186 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116188 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116190 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116193 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116195 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116198 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116200 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116202 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116205 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116207 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116211 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 14:52:32.117040 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116216 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116219 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116222 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116225 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116227 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116230 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116233 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116236 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116238 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116242 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116246 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116251 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116274 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116277 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116279 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116282 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116284 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116287 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116289 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116292 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 14:52:32.117535 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116294 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116296 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116299 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116302 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116304 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116307 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116309 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116312 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116314 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116316 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116319 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116321 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116323 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116326 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116328 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116331 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116333 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116336 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116338 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 14:52:32.118029 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116340 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116343 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116345 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116348 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116352 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116355 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116357 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116360 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116362 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116365 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116367 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116370 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116372 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116399 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116402 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116405 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116408 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116411 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116413 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116416 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 14:52:32.118544 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116418 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116421 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116424 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116426 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116429 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116431 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116433 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116436 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116440 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116443 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116446 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116449 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116451 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116454 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116457 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116459 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116465 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116468 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116834 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116839 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 14:52:32.119031 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116842 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116844 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116847 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116849 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116852 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116855 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116857 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116860 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116862 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116865 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116867 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116871 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116875 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116878 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116881 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116883 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116886 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116888 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116892 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 14:52:32.119550 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116896 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116898 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116901 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116904 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116906 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116909 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116912 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116914 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116917 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116920 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116923 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116926 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116928 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116931 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116933 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116935 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116938 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116940 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116942 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 14:52:32.120039 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116945 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116947 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116949 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116952 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116955 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116957 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116959 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116962 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116964 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116967 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116969 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116971 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116974 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116977 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116979 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116982 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116984 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116987 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116989 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 14:52:32.120517 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116992 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116994 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.116997 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117000 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117002 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117004 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117007 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117009 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117012 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117014 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117017 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117020 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117024 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117027 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117030 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117033 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117035 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117038 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117040 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117043 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 14:52:32.120975 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117045 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117048 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117051 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117054 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117056 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117059 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117061 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117064 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117066 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117068 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117071 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117073 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117075 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117078 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117080 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117082 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117085 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117087 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117089 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117092 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 14:52:32.121508 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117166 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117177 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117184 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117190 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117194 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117197 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117201 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117205 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117208 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117210 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117214 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117216 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117219 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117222 2582 flags.go:64] FLAG: --cgroup-root="" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117224 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117227 2582 flags.go:64] FLAG: --client-ca-file="" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117230 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117233 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117242 2582 flags.go:64] FLAG: --cluster-domain="" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117245 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117248 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117251 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117269 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117272 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 14:52:32.121982 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117275 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117278 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117281 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117284 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117287 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117290 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117293 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117295 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117299 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117302 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117305 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117308 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117310 2582 flags.go:64] FLAG: --enable-server="true" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117313 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117318 2582 flags.go:64] FLAG: --event-burst="100" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117321 2582 flags.go:64] FLAG: --event-qps="50" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117323 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117326 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117329 2582 flags.go:64] FLAG: --eviction-hard="" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117332 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117338 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117341 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117343 2582 flags.go:64] FLAG: --eviction-soft="" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117346 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117349 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 14:52:32.122579 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117354 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117357 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117360 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117363 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117365 2582 flags.go:64] FLAG: --feature-gates="" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117369 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117372 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117375 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117377 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117382 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117385 2582 flags.go:64] FLAG: --help="false" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117388 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117391 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117394 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117397 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117400 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117403 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117406 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117409 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117411 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117414 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117417 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117420 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117423 2582 flags.go:64] FLAG: --kube-reserved="" Jul 23 14:52:32.123199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117425 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117428 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117431 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117433 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117436 2582 flags.go:64] FLAG: --lock-file="" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117439 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117441 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117444 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117450 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117454 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117456 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117459 2582 flags.go:64] FLAG: --logging-format="text" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117461 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117464 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117467 2582 flags.go:64] FLAG: --manifest-url="" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117470 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117474 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117478 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117482 2582 flags.go:64] FLAG: --max-pods="110" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117484 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117487 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117490 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117493 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117495 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117499 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 14:52:32.123818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117501 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117508 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117511 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117514 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117517 2582 flags.go:64] FLAG: --pod-cidr="" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117520 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117525 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117528 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117530 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117533 2582 flags.go:64] FLAG: --port="10250" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117536 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117538 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c4af4d1d2a292764" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117541 2582 flags.go:64] FLAG: --qos-reserved="" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117545 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117548 2582 flags.go:64] FLAG: --register-node="true" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117551 2582 flags.go:64] FLAG: --register-with-taints="" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117556 2582 flags.go:64] FLAG: --registry-burst="10" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117559 2582 flags.go:64] FLAG: --registry-qps="5" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117562 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117564 2582 flags.go:64] FLAG: --reserved-memory="" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117568 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117570 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117573 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117576 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117580 2582 flags.go:64] FLAG: --runonce="false" Jul 23 14:52:32.124459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117583 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117585 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117588 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117591 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117594 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117597 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117600 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117603 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117605 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117608 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117611 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117614 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117616 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117619 2582 flags.go:64] FLAG: --system-cgroups="" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117622 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117627 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117630 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117632 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117637 2582 flags.go:64] FLAG: --tls-min-version="" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117639 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117642 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117645 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117648 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117652 2582 flags.go:64] FLAG: --v="2" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117655 2582 flags.go:64] FLAG: --version="false" Jul 23 14:52:32.125098 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117660 2582 flags.go:64] FLAG: --vmodule="" Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117664 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.117667 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117779 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117783 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117786 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117790 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117793 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117795 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117797 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117800 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117802 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117806 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117808 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117810 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117813 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117816 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117818 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117821 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117823 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 14:52:32.125741 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117826 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117828 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117832 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117836 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117839 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117842 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117844 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117847 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117850 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117853 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117857 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117859 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117862 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117865 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117868 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117870 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117873 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117877 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117881 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 14:52:32.126231 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117884 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117886 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117889 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117891 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117893 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117896 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117898 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117900 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117903 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117905 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117908 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117910 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117912 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117915 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117917 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117919 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117922 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117924 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117926 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117929 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 14:52:32.126722 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117931 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117933 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117936 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117940 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117943 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117945 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117948 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117950 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117953 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117955 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117958 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117961 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117965 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117968 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117971 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117974 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117977 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117979 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117981 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 14:52:32.127201 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117984 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117986 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117989 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117991 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117993 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117996 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.117998 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118001 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118003 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118005 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118008 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118010 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118012 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118015 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118017 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118020 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118023 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118026 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118028 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 14:52:32.127683 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118030 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118033 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118037 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118039 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.118042 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.118049 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.125367 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.125385 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.125405 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.125415 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125476 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125481 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125484 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125488 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 14:52:32.128136 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125492 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125496 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125499 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125502 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125505 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125507 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125510 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125512 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125515 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125518 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125520 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125523 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125525 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125528 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125530 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125533 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125536 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125538 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125541 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125543 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 14:52:32.128542 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125545 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125548 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125551 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125553 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125556 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125560 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125563 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125565 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125568 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125571 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125573 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125576 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125578 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125581 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125583 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125586 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125588 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125591 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125593 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125596 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 14:52:32.129069 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125598 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125601 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125605 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125608 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125611 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125614 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125616 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125619 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125621 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125624 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125627 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125629 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125632 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125634 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125637 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125639 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125642 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125645 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125653 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 14:52:32.129590 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125656 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125658 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125661 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125663 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125666 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125668 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125671 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125673 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125675 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125678 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125680 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125683 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125685 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125687 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125690 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125692 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125695 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125698 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125700 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125703 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 14:52:32.130045 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125705 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125708 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125710 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125713 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125715 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125717 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125720 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125723 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125725 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125727 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125730 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125732 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125741 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125744 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125748 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 14:52:32.130570 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125751 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.125756 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125896 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125900 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125903 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125907 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125910 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125913 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125916 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125919 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125922 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125924 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125927 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125930 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 14:52:32.130938 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125932 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125935 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125937 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125940 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125942 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125944 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125947 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125949 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125952 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125954 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125956 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125959 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125961 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125964 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125966 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125968 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125976 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125978 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125981 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125983 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 14:52:32.131309 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125985 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125988 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125990 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125992 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125995 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125997 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.125999 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126002 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126004 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126007 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126010 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126012 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126014 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126017 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126019 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126021 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126024 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126026 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126029 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126031 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 14:52:32.131795 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126034 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126036 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126038 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126041 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126043 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126061 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126065 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126068 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126071 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126080 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126083 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126085 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126088 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126091 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126094 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126096 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126099 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126101 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126103 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 14:52:32.132280 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126106 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126108 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126110 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126113 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126116 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126118 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126121 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126123 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126125 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126128 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126130 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126133 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126135 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126137 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126140 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126142 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126145 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126147 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126150 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 14:52:32.132781 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126153 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126156 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126159 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126161 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126168 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126171 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126173 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126176 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:32.126178 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.126182 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.126336 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.128625 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.129958 2582 server.go:1019] "Starting client certificate rotation" Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.130053 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 14:52:32.133247 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.130092 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 14:52:32.133616 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.132647 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 14:52:32.134496 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.134483 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 14:52:32.142971 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.142953 2582 log.go:25] "Validated CRI v1 runtime API" Jul 23 14:52:32.145124 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.145107 2582 log.go:25] "Validated CRI v1 image API" Jul 23 14:52:32.146599 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.146583 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 14:52:32.149079 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.149054 2582 fs.go:135] Filesystem UUIDs: map[774dc18a-78fa-43ad-ac9f-0374994bdd61:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9788cbf5-9eba-4e67-9185-3749e9598a6b:/dev/nvme0n1p4] Jul 23 14:52:32.149143 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.149079 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 14:52:32.154230 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.154120 2582 manager.go:216] Machine: {Timestamp:2026-07-23 14:52:32.15258261 +0000 UTC m=+0.304031970 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100129 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a644739706d1a23f6327ee19cec92 SystemUUID:ec2a6447-3970-6d1a-23f6-327ee19cec92 BootID:2d9d0b6c-fb1c-4170-83fd-763725135cd9 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:13:dd:fa:b0:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:13:dd:fa:b0:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:32:f8:f6:85:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 14:52:32.154230 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.154226 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 14:52:32.154401 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.154344 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 14:52:32.156931 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.156909 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 14:52:32.157064 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.156935 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 14:52:32.157594 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.157584 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 14:52:32.157631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.157597 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 14:52:32.157631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.157604 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 14:52:32.157631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.157614 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 14:52:32.159387 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.159377 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 14:52:32.162948 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.162937 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 23 14:52:32.163062 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163053 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 14:52:32.163095 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163086 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 23 14:52:32.163124 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163113 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 14:52:32.163152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163134 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 14:52:32.163152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163147 2582 kubelet.go:398] "Adding apiserver pod source" Jul 23 14:52:32.163203 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163161 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 14:52:32.163683 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163664 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 14:52:32.163751 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163681 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 14:52:32.163751 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.163687 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 14:52:32.165737 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.165715 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 14:52:32.168152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.168138 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 14:52:32.168208 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.168167 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 14:52:32.169485 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169474 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 14:52:32.169523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169490 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 14:52:32.169523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169496 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 14:52:32.169523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169501 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 14:52:32.169523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169509 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 14:52:32.169523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169515 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 14:52:32.169523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169523 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 14:52:32.169687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169528 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 14:52:32.169687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169535 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 14:52:32.169687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169546 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 14:52:32.169687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169562 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 14:52:32.169687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.169567 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 14:52:32.170296 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.170286 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 14:52:32.170326 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.170296 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 14:52:32.173583 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.173570 2582 server.go:1268] "Started kubelet" Jul 23 14:52:32.173680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.173657 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 14:52:32.173751 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.173680 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 14:52:32.173751 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.173699 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 14:52:32.174196 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.174178 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 14:52:32.174342 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.174219 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 14:52:32.174443 ip-10-0-142-11 systemd[1]: Started Kubernetes Kubelet. Jul 23 14:52:32.177251 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.177215 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 14:52:32.177418 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.177388 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 14:52:32.178925 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.178891 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 23 14:52:32.179401 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.179375 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 14:52:32.183584 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.182598 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-11.ec2.internal.18c4f2d5bde6eab9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-11.ec2.internal,UID:ip-10-0-142-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-11.ec2.internal,},FirstTimestamp:2026-07-23 14:52:32.173542073 +0000 UTC m=+0.324991433,LastTimestamp:2026-07-23 14:52:32.173542073 +0000 UTC m=+0.324991433,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-11.ec2.internal,}" Jul 23 14:52:32.188064 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188040 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 14:52:32.188132 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188088 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 14:52:32.188184 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.188159 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.188757 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188741 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 14:52:32.188840 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188761 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 14:52:32.188840 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188744 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 14:52:32.188957 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188868 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 23 14:52:32.188957 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188876 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 23 14:52:32.188957 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188907 2582 factory.go:55] Registering systemd factory Jul 23 14:52:32.188957 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.188921 2582 factory.go:223] Registration of the systemd container factory successfully Jul 23 14:52:32.189154 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.189141 2582 factory.go:153] Registering CRI-O factory Jul 23 14:52:32.189205 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.189157 2582 factory.go:223] Registration of the crio container factory successfully Jul 23 14:52:32.189283 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.189206 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 14:52:32.189283 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.189228 2582 factory.go:103] Registering Raw factory Jul 23 14:52:32.189283 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.189242 2582 manager.go:1196] Started watching for new ooms in manager Jul 23 14:52:32.189659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.189645 2582 manager.go:349] Starting recovery of all containers Jul 23 14:52:32.192666 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.192639 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 14:52:32.192752 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.192736 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 14:52:32.200984 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.200963 2582 manager.go:354] Recovery completed Jul 23 14:52:32.205860 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.205847 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.207871 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.207846 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.207961 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.207886 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.207961 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.207898 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.208426 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.208413 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 14:52:32.208426 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.208422 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.208551 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.208425 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 14:52:32.208551 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.208493 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 23 14:52:32.209832 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.209805 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.209921 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.209840 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.209921 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.209866 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.210055 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.210031 2582 policy_none.go:49] "None policy: Start" Jul 23 14:52:32.210055 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.210050 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 14:52:32.210154 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.210062 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 14:52:32.210329 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.210237 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-11.ec2.internal.18c4f2d5bff2be20 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-11.ec2.internal,UID:ip-10-0-142-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-11.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-11.ec2.internal,},FirstTimestamp:2026-07-23 14:52:32.20787152 +0000 UTC m=+0.359320882,LastTimestamp:2026-07-23 14:52:32.20787152 +0000 UTC m=+0.359320882,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-11.ec2.internal,}" Jul 23 14:52:32.211307 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.211250 2582 policy_none.go:47] "Start" Jul 23 14:52:32.214238 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.214220 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bwd58" Jul 23 14:52:32.217755 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.217661 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-11.ec2.internal.18c4f2d5bff30b5d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-11.ec2.internal,UID:ip-10-0-142-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-11.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-11.ec2.internal,},FirstTimestamp:2026-07-23 14:52:32.207891293 +0000 UTC m=+0.359340653,LastTimestamp:2026-07-23 14:52:32.207891293 +0000 UTC m=+0.359340653,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-11.ec2.internal,}" Jul 23 14:52:32.223024 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.223009 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bwd58" Jul 23 14:52:32.247838 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.247825 2582 manager.go:337] "Starting Device Plugin manager" Jul 23 14:52:32.247934 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.247856 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 14:52:32.247934 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.247865 2582 server.go:87] "Starting device plugin registration server" Jul 23 14:52:32.248110 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.248097 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 14:52:32.248176 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.248116 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 14:52:32.248591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.248288 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 14:52:32.248591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.248531 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 14:52:32.248591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.248541 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 14:52:32.248852 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.248837 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 14:52:32.248912 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.248889 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.285974 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.285942 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 14:52:32.287315 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.287292 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 14:52:32.287416 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.287331 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 14:52:32.287416 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.287356 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 14:52:32.287416 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.287398 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 14:52:32.290271 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.290235 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 14:52:32.348533 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.348494 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.349539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.349519 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.349612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.349558 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.349612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.349570 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.349612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.349593 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.372005 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.371954 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.372005 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.371981 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-11.ec2.internal\": node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.388329 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.388308 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal"] Jul 23 14:52:32.388403 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.388363 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.389152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.389138 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.389211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.389163 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.389211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.389166 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/306925b23e88a76b5c0fed85f97a3a0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal\" (UID: \"306925b23e88a76b5c0fed85f97a3a0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.389211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.389178 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.389211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.389186 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/306925b23e88a76b5c0fed85f97a3a0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal\" (UID: \"306925b23e88a76b5c0fed85f97a3a0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.390453 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.390443 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.390567 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.390555 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.390602 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.390581 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.391081 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.391067 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.391081 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.391077 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.391196 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.391093 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.391196 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.391098 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.391196 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.391108 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.391196 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.391108 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.392437 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.392424 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.392482 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.392449 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 14:52:32.393063 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.393043 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientMemory" Jul 23 14:52:32.393153 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.393075 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 14:52:32.393153 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.393087 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeHasSufficientPID" Jul 23 14:52:32.406384 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.406367 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.412852 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.412835 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-11.ec2.internal\" not found" node="ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.416964 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.416947 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-11.ec2.internal\" not found" node="ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.489788 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.489760 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/306925b23e88a76b5c0fed85f97a3a0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal\" (UID: \"306925b23e88a76b5c0fed85f97a3a0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.489788 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.489793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/306925b23e88a76b5c0fed85f97a3a0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal\" (UID: \"306925b23e88a76b5c0fed85f97a3a0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.489972 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.489838 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/306925b23e88a76b5c0fed85f97a3a0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal\" (UID: \"306925b23e88a76b5c0fed85f97a3a0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.489972 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.489864 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/306925b23e88a76b5c0fed85f97a3a0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal\" (UID: \"306925b23e88a76b5c0fed85f97a3a0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.506456 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.506431 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.590424 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.590381 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75c2663eec2141df3d5c4f1c1932094d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-11.ec2.internal\" (UID: \"75c2663eec2141df3d5c4f1c1932094d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.607528 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.607502 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.691183 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.691115 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75c2663eec2141df3d5c4f1c1932094d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-11.ec2.internal\" (UID: \"75c2663eec2141df3d5c4f1c1932094d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.691183 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.691174 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75c2663eec2141df3d5c4f1c1932094d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-11.ec2.internal\" (UID: \"75c2663eec2141df3d5c4f1c1932094d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.708299 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.708239 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.718421 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.718400 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.721224 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:32.721207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" Jul 23 14:52:32.809327 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.809290 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:32.909907 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:32.909867 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:33.010457 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:33.010362 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:33.110946 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:33.110903 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:33.130096 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.130067 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 14:52:33.130221 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.130205 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 14:52:33.211903 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:33.211867 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-11.ec2.internal\" not found" Jul 23 14:52:33.228350 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.228300 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 14:47:32 +0000 UTC" deadline="2028-01-11 09:56:36.689021733 +0000 UTC" Jul 23 14:52:33.228350 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.228326 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12883h4m3.460698756s" Jul 23 14:52:33.281559 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.281485 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 14:52:33.288983 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.288963 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" Jul 23 14:52:33.302681 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.302657 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 14:52:33.304411 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.304389 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" Jul 23 14:52:33.311584 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.311569 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 14:52:33.325769 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:33.325741 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306925b23e88a76b5c0fed85f97a3a0d.slice/crio-85ad44d02759c282945152d054f06aa8ee43b8d4a6f5d7595ebfaff3f0191467 WatchSource:0}: Error finding container 85ad44d02759c282945152d054f06aa8ee43b8d4a6f5d7595ebfaff3f0191467: Status 404 returned error can't find the container with id 85ad44d02759c282945152d054f06aa8ee43b8d4a6f5d7595ebfaff3f0191467 Jul 23 14:52:33.326474 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:33.326452 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c2663eec2141df3d5c4f1c1932094d.slice/crio-48ce2c250d6109fc5e092837edb92d5d88ad57f7f56848bce6bee9af72e4bdb7 WatchSource:0}: Error finding container 48ce2c250d6109fc5e092837edb92d5d88ad57f7f56848bce6bee9af72e4bdb7: Status 404 returned error can't find the container with id 48ce2c250d6109fc5e092837edb92d5d88ad57f7f56848bce6bee9af72e4bdb7 Jul 23 14:52:33.330451 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.330426 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 14:52:33.633987 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.633892 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 14:52:33.639624 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:33.639606 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 14:52:34.164819 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.164784 2582 apiserver.go:52] "Watching apiserver" Jul 23 14:52:34.174153 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.174126 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 14:52:34.177091 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.177064 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-2svvx","kube-system/global-pull-secret-syncer-f2fgw","kube-system/konnectivity-agent-nqjm5","openshift-cluster-node-tuning-operator/tuned-ctr8r","openshift-image-registry/node-ca-8pjfm","openshift-multus/multus-additional-cni-plugins-rqkgc","openshift-multus/multus-rfwvt","openshift-network-diagnostics/network-check-target-c6nf8","openshift-network-operator/iptables-alerter-6c6ts","kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w","openshift-dns/node-resolver-mfddd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal","openshift-multus/network-metrics-daemon-xjwws"] Jul 23 14:52:34.178963 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.178939 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.180739 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.180719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.182234 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.182214 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.182414 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.182391 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 14:52:34.182600 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.182583 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.182886 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.182868 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-d2tcz\"" Jul 23 14:52:34.182959 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.182900 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 14:52:34.183060 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.183044 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.184363 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.184345 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.185174 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.185156 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 14:52:34.185264 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.185205 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.185447 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.185342 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wd46w\"" Jul 23 14:52:34.185610 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.185594 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 14:52:34.185802 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.185786 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.186199 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.186177 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gbr2c\"" Jul 23 14:52:34.186321 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.186289 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 14:52:34.186607 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.186590 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.187007 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.186990 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.187654 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.187628 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.187750 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.187700 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9rbqv\"" Jul 23 14:52:34.188517 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.188498 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 14:52:34.189323 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.189304 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.189906 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.189724 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x8f5b\"" Jul 23 14:52:34.189971 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.189954 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.189971 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.189964 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 14:52:34.191198 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.191177 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.192856 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.192838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.194178 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.194158 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 14:52:34.194437 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.194421 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.194520 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.194498 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:34.194886 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.194869 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 14:52:34.195051 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.194983 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jdfqr\"" Jul 23 14:52:34.195587 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.195568 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.195675 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.195614 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 14:52:34.196195 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.196003 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.196195 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.196061 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:34.196354 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.196282 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.196354 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.196344 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q94wp\"" Jul 23 14:52:34.198123 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.198107 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:34.198215 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.198163 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.199969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-socket-dir-parent\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-hostroot\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a942429-26a7-434f-ba0f-8d7ec0691b1c-konnectivity-ca\") pod \"konnectivity-agent-nqjm5\" (UID: \"3a942429-26a7-434f-ba0f-8d7ec0691b1c\") " pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-sys\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200110 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-host\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200139 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5g94\" (UniqueName: \"kubernetes.io/projected/bb31c270-e90d-4a01-99e1-5c3d39e63631-kube-api-access-z5g94\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200180 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-system-cni-dir\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200210 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74d8f36b-9973-4647-a699-1c3ccf7fd19c-cni-binary-copy\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-conf-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysconfig\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysctl-d\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.200425 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-lib-modules\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-multus-certs\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-socket-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200632 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzhwg\" (UniqueName: \"kubernetes.io/projected/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-kube-api-access-qzhwg\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysctl-conf\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-cnibin\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200722 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200781 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200819 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-system-cni-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-modprobe-d\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200884 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-systemd\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb31c270-e90d-4a01-99e1-5c3d39e63631-host\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200967 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb31c270-e90d-4a01-99e1-5c3d39e63631-serviceca\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.201001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.200994 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-cnibin\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201023 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-os-release\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201080 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-sys-fs\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-kubernetes\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-var-lib-kubelet\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201164 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dthn\" (UniqueName: \"kubernetes.io/projected/29f61377-df3b-438e-b572-8648f23ab8df-kube-api-access-4dthn\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-kubelet\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201228 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-registration-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201280 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29f61377-df3b-438e-b572-8648f23ab8df-etc-tuned\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29f61377-df3b-438e-b572-8648f23ab8df-tmp\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201344 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-os-release\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201416 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8p4s\" (UniqueName: \"kubernetes.io/projected/f5a883e7-68b9-4878-af6d-68a66b399ce3-kube-api-access-j8p4s\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-k8s-cni-cncf-io\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-daemon-config\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.201691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201534 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-run\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201562 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a942429-26a7-434f-ba0f-8d7ec0691b1c-agent-certs\") pod \"konnectivity-agent-nqjm5\" (UID: \"3a942429-26a7-434f-ba0f-8d7ec0691b1c\") " pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201602 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-cni-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201626 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-netns\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201656 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-cni-bin\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-cni-multus\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-etc-kubernetes\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201741 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmg7\" (UniqueName: \"kubernetes.io/projected/74d8f36b-9973-4647-a699-1c3ccf7fd19c-kube-api-access-jdmg7\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.202428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.201773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-device-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.204910 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.204888 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 14:52:34.209562 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.209503 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.211009 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.210987 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.212534 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.212060 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.212534 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.212232 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 14:52:34.212534 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.212365 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.213522 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.213504 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 14:52:34.213748 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.213733 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 14:52:34.213748 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.213741 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 14:52:34.213922 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.213908 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 14:52:34.214085 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.214071 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4m8j4\"" Jul 23 14:52:34.214218 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.214207 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-npssl\"" Jul 23 14:52:34.214398 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.214381 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 14:52:34.228707 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.228686 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p9fxt" Jul 23 14:52:34.237293 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.237193 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p9fxt" Jul 23 14:52:34.290539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.290511 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 14:52:34.292817 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.292769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" event={"ID":"306925b23e88a76b5c0fed85f97a3a0d","Type":"ContainerStarted","Data":"85ad44d02759c282945152d054f06aa8ee43b8d4a6f5d7595ebfaff3f0191467"} Jul 23 14:52:34.294457 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.294428 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" event={"ID":"75c2663eec2141df3d5c4f1c1932094d","Type":"ContainerStarted","Data":"48ce2c250d6109fc5e092837edb92d5d88ad57f7f56848bce6bee9af72e4bdb7"} Jul 23 14:52:34.302844 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.302797 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-lib-modules\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.302945 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.302860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-node-log\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.302945 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.302896 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-host-slash\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.302945 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.302920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d34f5e56-e8da-4a6a-9356-84a4ef68174a-tmp-dir\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.303099 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.302953 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-multus-certs\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.303099 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.302986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-socket-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.303099 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzhwg\" (UniqueName: \"kubernetes.io/projected/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-kube-api-access-qzhwg\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.303099 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303035 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkdfm\" (UniqueName: \"kubernetes.io/projected/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-kube-api-access-rkdfm\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.303099 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303058 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-system-cni-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.303099 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-systemd\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303104 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb31c270-e90d-4a01-99e1-5c3d39e63631-serviceca\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303128 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303179 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-os-release\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303202 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303227 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-sys-fs\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303291 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-kubernetes\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dthn\" (UniqueName: \"kubernetes.io/projected/29f61377-df3b-438e-b572-8648f23ab8df-kube-api-access-4dthn\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-os-release\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303407 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-var-lib-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-registration-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303480 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8p4s\" (UniqueName: \"kubernetes.io/projected/f5a883e7-68b9-4878-af6d-68a66b399ce3-kube-api-access-j8p4s\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-run-netns\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.303554 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303530 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-etc-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303561 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-ovn\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-log-socket\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp24l\" (UniqueName: \"kubernetes.io/projected/3d60ecea-1743-434c-8b1c-78b027daac9b-kube-api-access-sp24l\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-k8s-cni-cncf-io\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303674 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-daemon-config\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-env-overrides\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303735 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz999\" (UniqueName: \"kubernetes.io/projected/d34f5e56-e8da-4a6a-9356-84a4ef68174a-kube-api-access-cz999\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-cnibin\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303869 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-cni-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-netns\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-cni-multus\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303946 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmg7\" (UniqueName: \"kubernetes.io/projected/74d8f36b-9973-4647-a699-1c3ccf7fd19c-kube-api-access-jdmg7\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.303977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d34f5e56-e8da-4a6a-9356-84a4ef68174a-hosts-file\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-systemd\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.304211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovnkube-config\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-conf-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304099 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysctl-d\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304152 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-iptables-alerter-script\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysctl-conf\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304198 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304220 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovnkube-script-lib\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304243 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65cx7\" (UniqueName: \"kubernetes.io/projected/0181f318-c751-4ab6-bac0-cae5f18f87f0-kube-api-access-65cx7\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-modprobe-d\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb31c270-e90d-4a01-99e1-5c3d39e63631-host\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304399 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b8ac723b-5a09-485f-b5e9-faf6eef37039-kubelet-config\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304453 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b8ac723b-5a09-485f-b5e9-faf6eef37039-dbus\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304484 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-kubelet\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304526 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-cnibin\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304573 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-var-lib-kubelet\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305030 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304603 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-systemd-units\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304629 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304657 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-kubelet\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304680 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29f61377-df3b-438e-b572-8648f23ab8df-etc-tuned\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304715 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29f61377-df3b-438e-b572-8648f23ab8df-tmp\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304751 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304775 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-cni-bin\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovn-node-metrics-cert\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304829 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-run\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304963 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a942429-26a7-434f-ba0f-8d7ec0691b1c-konnectivity-ca\") pod \"konnectivity-agent-nqjm5\" (UID: \"3a942429-26a7-434f-ba0f-8d7ec0691b1c\") " pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.304990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-host\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305028 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-system-cni-dir\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305052 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a942429-26a7-434f-ba0f-8d7ec0691b1c-agent-certs\") pod \"konnectivity-agent-nqjm5\" (UID: \"3a942429-26a7-434f-ba0f-8d7ec0691b1c\") " pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-cni-bin\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305110 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-etc-kubernetes\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.305826 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305150 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-device-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-cni-netd\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-socket-dir-parent\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305232 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-hostroot\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-sys\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305300 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5g94\" (UniqueName: \"kubernetes.io/projected/bb31c270-e90d-4a01-99e1-5c3d39e63631-kube-api-access-z5g94\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305322 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-slash\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74d8f36b-9973-4647-a699-1c3ccf7fd19c-cni-binary-copy\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysconfig\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305524 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysconfig\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-lib-modules\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305772 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-multus-certs\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.305897 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-socket-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.306217 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-system-cni-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.306638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.306292 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-systemd\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.307307 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.306728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb31c270-e90d-4a01-99e1-5c3d39e63631-serviceca\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.307307 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.306843 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-os-release\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.307307 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.306894 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.307307 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.306942 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-sys-fs\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.307307 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.307012 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-kubernetes\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.307650 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.307278 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-os-release\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.307778 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.307753 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb31c270-e90d-4a01-99e1-5c3d39e63631-host\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.307848 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.307781 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-registration-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.307942 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.307849 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-cnibin\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309609 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-hostroot\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309618 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309704 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-socket-dir-parent\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309779 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-system-cni-dir\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309858 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-k8s-cni-cncf-io\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309954 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-run-netns\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309964 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-host\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309965 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a942429-26a7-434f-ba0f-8d7ec0691b1c-konnectivity-ca\") pod \"konnectivity-agent-nqjm5\" (UID: \"3a942429-26a7-434f-ba0f-8d7ec0691b1c\") " pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.309989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-cnibin\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310096 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5a883e7-68b9-4878-af6d-68a66b399ce3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310114 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74d8f36b-9973-4647-a699-1c3ccf7fd19c-cni-binary-copy\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310162 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-modprobe-d\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310167 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-kubelet\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310205 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-conf-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-cni-dir\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310308 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-cni-multus\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310340 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74d8f36b-9973-4647-a699-1c3ccf7fd19c-multus-daemon-config\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.310859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310410 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-device-dir\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310523 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysctl-d\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310530 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-sys\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-var-lib-kubelet\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310722 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-host-var-lib-cni-bin\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.310772 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-run\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.311294 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/29f61377-df3b-438e-b572-8648f23ab8df-etc-sysctl-conf\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.311432 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74d8f36b-9973-4647-a699-1c3ccf7fd19c-etc-kubernetes\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.311680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.311646 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-cni-binary-copy\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.312105 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.312075 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5a883e7-68b9-4878-af6d-68a66b399ce3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.319118 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.318755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29f61377-df3b-438e-b572-8648f23ab8df-tmp\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.319118 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.319048 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a942429-26a7-434f-ba0f-8d7ec0691b1c-agent-certs\") pod \"konnectivity-agent-nqjm5\" (UID: \"3a942429-26a7-434f-ba0f-8d7ec0691b1c\") " pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.319780 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.319758 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5g94\" (UniqueName: \"kubernetes.io/projected/bb31c270-e90d-4a01-99e1-5c3d39e63631-kube-api-access-z5g94\") pod \"node-ca-8pjfm\" (UID: \"bb31c270-e90d-4a01-99e1-5c3d39e63631\") " pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.320734 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.320406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/29f61377-df3b-438e-b572-8648f23ab8df-etc-tuned\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.320734 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.320684 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzhwg\" (UniqueName: \"kubernetes.io/projected/3be5364f-8f5b-48a1-bcd8-cecedcf0a176-kube-api-access-qzhwg\") pod \"aws-ebs-csi-driver-node-lzg7w\" (UID: \"3be5364f-8f5b-48a1-bcd8-cecedcf0a176\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.320845 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.320764 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dthn\" (UniqueName: \"kubernetes.io/projected/29f61377-df3b-438e-b572-8648f23ab8df-kube-api-access-4dthn\") pod \"tuned-ctr8r\" (UID: \"29f61377-df3b-438e-b572-8648f23ab8df\") " pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.322470 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.322449 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmg7\" (UniqueName: \"kubernetes.io/projected/74d8f36b-9973-4647-a699-1c3ccf7fd19c-kube-api-access-jdmg7\") pod \"multus-rfwvt\" (UID: \"74d8f36b-9973-4647-a699-1c3ccf7fd19c\") " pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.322545 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.322484 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8p4s\" (UniqueName: \"kubernetes.io/projected/f5a883e7-68b9-4878-af6d-68a66b399ce3-kube-api-access-j8p4s\") pod \"multus-additional-cni-plugins-rqkgc\" (UID: \"f5a883e7-68b9-4878-af6d-68a66b399ce3\") " pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.405749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-node-log\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.405907 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405761 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-host-slash\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.405907 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d34f5e56-e8da-4a6a-9356-84a4ef68174a-tmp-dir\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.405907 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405814 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkdfm\" (UniqueName: \"kubernetes.io/projected/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-kube-api-access-rkdfm\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.405907 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-node-log\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.405907 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405842 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:34.405907 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405898 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405915 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-host-slash\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405933 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-var-lib-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-run-netns\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.405986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-etc-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-ovn\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-log-socket\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406087 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp24l\" (UniqueName: \"kubernetes.io/projected/3d60ecea-1743-434c-8b1c-78b027daac9b-kube-api-access-sp24l\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-env-overrides\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz999\" (UniqueName: \"kubernetes.io/projected/d34f5e56-e8da-4a6a-9356-84a4ef68174a-kube-api-access-cz999\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406151 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d34f5e56-e8da-4a6a-9356-84a4ef68174a-hosts-file\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406172 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-systemd\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovnkube-config\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-iptables-alerter-script\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d34f5e56-e8da-4a6a-9356-84a4ef68174a-tmp-dir\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovnkube-script-lib\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65cx7\" (UniqueName: \"kubernetes.io/projected/0181f318-c751-4ab6-bac0-cae5f18f87f0-kube-api-access-65cx7\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b8ac723b-5a09-485f-b5e9-faf6eef37039-kubelet-config\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406393 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b8ac723b-5a09-485f-b5e9-faf6eef37039-dbus\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-kubelet\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-systemd-units\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406514 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-cni-bin\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406538 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovn-node-metrics-cert\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406554 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d34f5e56-e8da-4a6a-9356-84a4ef68174a-hosts-file\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-cni-netd\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-slash\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.406726 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406724 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-slash\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407391 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406737 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407391 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406777 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-var-lib-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407391 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.406809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-systemd\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407391 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407328 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovnkube-config\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407391 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407372 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-iptables-alerter-script\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.407619 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407381 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-run-netns\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407619 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407399 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-etc-openvswitch\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407619 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407422 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-run-ovn\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407619 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407456 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-log-socket\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407913 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407894 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovnkube-script-lib\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.407983 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.407973 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.408278 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.408369 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:52:34.908341648 +0000 UTC m=+3.059791017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408458 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b8ac723b-5a09-485f-b5e9-faf6eef37039-kubelet-config\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408520 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b8ac723b-5a09-485f-b5e9-faf6eef37039-dbus\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408571 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-systemd-units\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408576 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-cni-bin\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408624 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-kubelet\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.408656 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408675 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0181f318-c751-4ab6-bac0-cae5f18f87f0-host-cni-netd\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.408763 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.408703 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:52:34.908686084 +0000 UTC m=+3.060135445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:34.409386 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.408770 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0181f318-c751-4ab6-bac0-cae5f18f87f0-env-overrides\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.412137 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.412111 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0181f318-c751-4ab6-bac0-cae5f18f87f0-ovn-node-metrics-cert\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.413499 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.413479 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:52:34.413650 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.413506 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:52:34.413650 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.413520 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:34.413650 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.413589 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:52:34.913573714 +0000 UTC m=+3.065023064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:34.417389 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.417338 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp24l\" (UniqueName: \"kubernetes.io/projected/3d60ecea-1743-434c-8b1c-78b027daac9b-kube-api-access-sp24l\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.417916 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.417896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65cx7\" (UniqueName: \"kubernetes.io/projected/0181f318-c751-4ab6-bac0-cae5f18f87f0-kube-api-access-65cx7\") pod \"ovnkube-node-2svvx\" (UID: \"0181f318-c751-4ab6-bac0-cae5f18f87f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.418622 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.418599 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz999\" (UniqueName: \"kubernetes.io/projected/d34f5e56-e8da-4a6a-9356-84a4ef68174a-kube-api-access-cz999\") pod \"node-resolver-mfddd\" (UID: \"d34f5e56-e8da-4a6a-9356-84a4ef68174a\") " pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.420283 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.419641 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkdfm\" (UniqueName: \"kubernetes.io/projected/15fd95ef-9ffd-4eaf-a1e0-8463fc986e43-kube-api-access-rkdfm\") pod \"iptables-alerter-6c6ts\" (UID: \"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43\") " pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.496059 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.496010 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rfwvt" Jul 23 14:52:34.511562 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.511538 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" Jul 23 14:52:34.521428 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.521406 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:34.530984 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.530964 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" Jul 23 14:52:34.538767 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.538745 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8pjfm" Jul 23 14:52:34.548408 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.548387 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" Jul 23 14:52:34.557180 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.557151 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6c6ts" Jul 23 14:52:34.564897 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.564878 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:52:34.572676 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.572656 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mfddd" Jul 23 14:52:34.709828 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.709749 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 14:52:34.909657 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.909625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:34.909820 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:34.909665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:34.909820 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.909802 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:34.909909 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.909891 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:34.909943 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.909903 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:52:35.909886762 +0000 UTC m=+4.061336125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:34.909980 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:34.909952 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:52:35.909940253 +0000 UTC m=+4.061389600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:35.010610 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:35.010525 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:35.010775 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.010724 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:52:35.010775 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.010751 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:52:35.010775 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.010764 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:35.010943 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.010828 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:52:36.010808414 +0000 UTC m=+4.162257765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:35.239276 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:35.239227 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 14:47:34 +0000 UTC" deadline="2028-01-10 07:12:38.486520754 +0000 UTC" Jul 23 14:52:35.239276 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:35.239272 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12856h20m3.247252565s" Jul 23 14:52:35.288640 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:35.288555 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:35.288799 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.288695 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:35.369235 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:35.369203 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34f5e56_e8da_4a6a_9356_84a4ef68174a.slice/crio-a3c4cf1a9d90c92f87e21b08a7b4aab9ffb6218b41ebcbd6dab2284cb2330711 WatchSource:0}: Error finding container a3c4cf1a9d90c92f87e21b08a7b4aab9ffb6218b41ebcbd6dab2284cb2330711: Status 404 returned error can't find the container with id a3c4cf1a9d90c92f87e21b08a7b4aab9ffb6218b41ebcbd6dab2284cb2330711 Jul 23 14:52:35.387345 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:35.387206 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f61377_df3b_438e_b572_8648f23ab8df.slice/crio-edd73f2b23d13ed35882951df22dfb0c7035705b13d6e9b21e24ce8f17b6f649 WatchSource:0}: Error finding container edd73f2b23d13ed35882951df22dfb0c7035705b13d6e9b21e24ce8f17b6f649: Status 404 returned error can't find the container with id edd73f2b23d13ed35882951df22dfb0c7035705b13d6e9b21e24ce8f17b6f649 Jul 23 14:52:35.389436 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:35.389411 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fd95ef_9ffd_4eaf_a1e0_8463fc986e43.slice/crio-bf765b140d6e0b35d647d605ecb5f7b6acbebc8df8f036ee31cf9a86259c4e12 WatchSource:0}: Error finding container bf765b140d6e0b35d647d605ecb5f7b6acbebc8df8f036ee31cf9a86259c4e12: Status 404 returned error can't find the container with id bf765b140d6e0b35d647d605ecb5f7b6acbebc8df8f036ee31cf9a86259c4e12 Jul 23 14:52:35.392514 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:35.392491 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d8f36b_9973_4647_a699_1c3ccf7fd19c.slice/crio-b9c26aeb24051bdd9cedc5972e065f33ea70614f681789dbcd80c47e34bfdfc8 WatchSource:0}: Error finding container b9c26aeb24051bdd9cedc5972e065f33ea70614f681789dbcd80c47e34bfdfc8: Status 404 returned error can't find the container with id b9c26aeb24051bdd9cedc5972e065f33ea70614f681789dbcd80c47e34bfdfc8 Jul 23 14:52:35.398669 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:52:35.398585 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a942429_26a7_434f_ba0f_8d7ec0691b1c.slice/crio-ec58e34848efaaaeb6d6111a3ed1db55aed9538b393f880f7cba14c2fecfefc2 WatchSource:0}: Error finding container ec58e34848efaaaeb6d6111a3ed1db55aed9538b393f880f7cba14c2fecfefc2: Status 404 returned error can't find the container with id ec58e34848efaaaeb6d6111a3ed1db55aed9538b393f880f7cba14c2fecfefc2 Jul 23 14:52:35.918730 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:35.918458 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:35.918854 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:35.918740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:35.918854 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.918605 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:35.918854 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.918806 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:35.918854 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.918831 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:52:37.918813939 +0000 UTC m=+6.070263292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:35.918854 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:35.918847 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:52:37.918836595 +0000 UTC m=+6.070285957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:36.019349 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.019284 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:36.019520 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:36.019465 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:52:36.019520 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:36.019493 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:52:36.019520 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:36.019505 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:36.019667 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:36.019563 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:52:38.019543886 +0000 UTC m=+6.170993234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:36.240128 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.240019 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 14:47:34 +0000 UTC" deadline="2028-04-10 18:26:34.963938454 +0000 UTC" Jul 23 14:52:36.240128 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.240080 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15051h33m58.72386206s" Jul 23 14:52:36.290716 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.290686 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:36.290876 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:36.290832 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:36.291272 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.291233 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:36.291379 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:36.291356 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:36.310573 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.310462 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" event={"ID":"29f61377-df3b-438e-b572-8648f23ab8df","Type":"ContainerStarted","Data":"edd73f2b23d13ed35882951df22dfb0c7035705b13d6e9b21e24ce8f17b6f649"} Jul 23 14:52:36.325460 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.325353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mfddd" event={"ID":"d34f5e56-e8da-4a6a-9356-84a4ef68174a","Type":"ContainerStarted","Data":"a3c4cf1a9d90c92f87e21b08a7b4aab9ffb6218b41ebcbd6dab2284cb2330711"} Jul 23 14:52:36.347551 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.347514 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerStarted","Data":"41eafac9faae19ae5812311657ea123df4b2d5e5f3b17c40abb34ee446c27cce"} Jul 23 14:52:36.356742 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.356709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8pjfm" event={"ID":"bb31c270-e90d-4a01-99e1-5c3d39e63631","Type":"ContainerStarted","Data":"b2ba3f9e3747b05b57711da42b2d03b95e653c9be32582cfeff45172fbe5dfc9"} Jul 23 14:52:36.372373 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.372334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6c6ts" event={"ID":"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43","Type":"ContainerStarted","Data":"bf765b140d6e0b35d647d605ecb5f7b6acbebc8df8f036ee31cf9a86259c4e12"} Jul 23 14:52:36.379674 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.379079 2582 generic.go:356] "Generic (PLEG): container finished" podID="306925b23e88a76b5c0fed85f97a3a0d" containerID="f9f3574852f1da65fdf317f535336ed1aaae359d1593ebfa8ebd21fd892ba35d" exitCode=0 Jul 23 14:52:36.379674 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.379409 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" event={"ID":"306925b23e88a76b5c0fed85f97a3a0d","Type":"ContainerDied","Data":"f9f3574852f1da65fdf317f535336ed1aaae359d1593ebfa8ebd21fd892ba35d"} Jul 23 14:52:36.407019 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.406979 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" event={"ID":"75c2663eec2141df3d5c4f1c1932094d","Type":"ContainerStarted","Data":"c0be4085a2a7e12b9915ad3e9e4ab5742385b1956e8a51cf5e89f064c40bf492"} Jul 23 14:52:36.418078 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.417990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nqjm5" event={"ID":"3a942429-26a7-434f-ba0f-8d7ec0691b1c","Type":"ContainerStarted","Data":"ec58e34848efaaaeb6d6111a3ed1db55aed9538b393f880f7cba14c2fecfefc2"} Jul 23 14:52:36.442763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.442663 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" event={"ID":"3be5364f-8f5b-48a1-bcd8-cecedcf0a176","Type":"ContainerStarted","Data":"9def3d41d63969a388d52aca5787fa10eb6a0711d5bffe5d1eaa9015a535e2ba"} Jul 23 14:52:36.461789 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.461750 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rfwvt" event={"ID":"74d8f36b-9973-4647-a699-1c3ccf7fd19c","Type":"ContainerStarted","Data":"b9c26aeb24051bdd9cedc5972e065f33ea70614f681789dbcd80c47e34bfdfc8"} Jul 23 14:52:36.469163 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:36.469127 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"fde56e4ba4a394b166f1b4cb137f918c7a5713091ebd4ee2cafa62503c31c157"} Jul 23 14:52:37.288398 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:37.288363 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:37.288982 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:37.288501 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:37.496353 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:37.496320 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" event={"ID":"306925b23e88a76b5c0fed85f97a3a0d","Type":"ContainerStarted","Data":"86c8a435d8ba24d5ebbe18460e5049a7a4ca5a8578be70b4bb5bcde7f614f160"} Jul 23 14:52:37.515277 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:37.514776 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-11.ec2.internal" podStartSLOduration=4.5147521600000005 podStartE2EDuration="4.51475216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:36.427058083 +0000 UTC m=+4.578507453" watchObservedRunningTime="2026-07-23 14:52:37.51475216 +0000 UTC m=+5.666201534" Jul 23 14:52:37.937378 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:37.936442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:37.937378 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:37.936545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:37.937378 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:37.936666 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:37.937378 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:37.936728 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:52:41.936709531 +0000 UTC m=+10.088158892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:37.937378 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:37.937113 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:37.937378 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:37.937159 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:52:41.937144987 +0000 UTC m=+10.088594367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:38.038110 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:38.037340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:38.038110 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:38.037585 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:52:38.038110 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:38.037607 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:52:38.038110 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:38.037619 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:38.038110 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:38.037678 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:52:42.037660093 +0000 UTC m=+10.189109444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:38.289434 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:38.289359 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:38.289434 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:38.289404 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:38.289962 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:38.289493 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:38.289962 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:38.289664 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:39.287848 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:39.287811 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:39.288043 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:39.287953 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:40.290573 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:40.290334 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:40.290573 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:40.290441 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:40.290573 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:40.290532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:40.291010 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:40.290619 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:41.288058 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:41.288025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:41.288225 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:41.288155 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:41.969602 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:41.968975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:41.969602 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:41.969073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:41.969602 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:41.969194 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:41.969602 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:41.969251 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:52:49.969232946 +0000 UTC m=+18.120682307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:41.969602 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:41.969388 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:41.969602 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:41.969449 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:52:49.969431995 +0000 UTC m=+18.120881349 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:42.069999 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:42.069962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:42.070187 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:42.070119 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:52:42.070187 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:42.070143 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:52:42.070187 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:42.070154 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:42.070365 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:42.070218 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:52:50.07019846 +0000 UTC m=+18.221647829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:42.289024 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:42.288906 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:42.289189 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:42.289042 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:42.289489 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:42.289470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:42.289608 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:42.289587 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:43.287855 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:43.287821 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:43.288337 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:43.287937 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:44.288653 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:44.288619 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:44.289098 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:44.288748 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:44.289180 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:44.289157 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:44.289369 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:44.289324 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:45.288803 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:45.288270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:45.288803 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:45.288469 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:46.288567 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:46.288532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:46.288567 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:46.288554 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:46.288814 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:46.288688 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:46.288874 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:46.288806 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:47.288101 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:47.288064 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:47.288315 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:47.288181 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:48.288249 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:48.287722 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:48.288249 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:48.287851 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:48.288801 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:48.288336 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:48.288801 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:48.288449 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:49.287817 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:49.287783 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:49.288097 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:49.287902 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:50.039688 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:50.039650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:50.040124 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:50.039705 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:50.040124 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.039829 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:50.040124 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.039832 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:50.040124 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.039882 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:06.039868712 +0000 UTC m=+34.191318059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:52:50.040124 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.039895 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:53:06.039888767 +0000 UTC m=+34.191338113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:52:50.140580 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:50.140544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:50.140755 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.140737 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:52:50.140817 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.140766 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:52:50.140817 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.140778 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:50.140904 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.140868 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:53:06.140824161 +0000 UTC m=+34.292273527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:52:50.288432 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:50.288404 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:50.288588 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.288544 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:50.288653 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:50.288588 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:50.288701 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:50.288674 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:51.287864 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:51.287824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:51.288301 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:51.287956 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:52.288158 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:52.288125 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:52.288628 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:52.288223 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:52.288628 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:52.288326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:52.288628 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:52.288414 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:53.287948 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:53.287916 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:53.288111 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:53.288020 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:54.288053 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:54.288009 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:54.288496 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:54.288009 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:54.288496 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:54.288136 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:54.288496 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:54.288181 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:55.288891 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.288438 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:55.289643 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:55.288991 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:55.552503 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.552469 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nqjm5" event={"ID":"3a942429-26a7-434f-ba0f-8d7ec0691b1c","Type":"ContainerStarted","Data":"07a622153eb1f2205eb852ecf340fd2a286ccd18d285d3c3e9cdeffa9957510f"} Jul 23 14:52:55.553904 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.553879 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" event={"ID":"3be5364f-8f5b-48a1-bcd8-cecedcf0a176","Type":"ContainerStarted","Data":"430092163ae385cde5236dceaf4697b073c58cd9069e5a834c141cdc4df07845"} Jul 23 14:52:55.555201 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.555180 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rfwvt" event={"ID":"74d8f36b-9973-4647-a699-1c3ccf7fd19c","Type":"ContainerStarted","Data":"d00c705dff20b96e2ea809b593dcd056d8404e5282b2cb2d91e46df0fb2a9256"} Jul 23 14:52:55.558111 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.558082 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"baf7e7b8b2d9ef4260235654ef715b08456830923d9908a5124ef3e78e89d127"} Jul 23 14:52:55.558212 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.558155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"2a56fbb8f9410ca7713b87cca9f95d88480badec8ecb9cbe83119199380e53c6"} Jul 23 14:52:55.558212 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.558208 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"dc2e0b25621477dc4cf51c85625f5002ace18c8743cf49864ace667a5072f818"} Jul 23 14:52:55.558368 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.558275 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"0a6c2686b5632c5d96a705121a42fecd0053f1a7a2b7fb58c85f7aade34c5a67"} Jul 23 14:52:55.558368 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.558327 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"9b65718a5d888fd5e047153b501feaabd1712f01e452df9f0957256121b961bb"} Jul 23 14:52:55.558454 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.558375 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"9f33497dd3c23524dae8c23e9fcc56c7327f888bf0d104d7a2503ca54aa29351"} Jul 23 14:52:55.559768 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.559742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" event={"ID":"29f61377-df3b-438e-b572-8648f23ab8df","Type":"ContainerStarted","Data":"a3e7d17e498828a2805adf5b03e308706ad1f0a834df97a27821fc8d542153ba"} Jul 23 14:52:55.561156 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.561133 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mfddd" event={"ID":"d34f5e56-e8da-4a6a-9356-84a4ef68174a","Type":"ContainerStarted","Data":"c40f903de8e3083ff43eaf52cc8f57a30f257d59d7942795ae8a8a6f17b9fdf7"} Jul 23 14:52:55.562567 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.562547 2582 generic.go:356] "Generic (PLEG): container finished" podID="f5a883e7-68b9-4878-af6d-68a66b399ce3" containerID="4d1f4c82785a242adda7b6e0b11a4eff8b4f02b6dc1f1a9588b49cd5256d23f8" exitCode=0 Jul 23 14:52:55.562650 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.562602 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerDied","Data":"4d1f4c82785a242adda7b6e0b11a4eff8b4f02b6dc1f1a9588b49cd5256d23f8"} Jul 23 14:52:55.564335 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.564303 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8pjfm" event={"ID":"bb31c270-e90d-4a01-99e1-5c3d39e63631","Type":"ContainerStarted","Data":"ff58687d56619e8a5fd58dd299cf1ecaec7de97faa77da0ebee09e41148bb811"} Jul 23 14:52:55.570782 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.570741 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nqjm5" podStartSLOduration=12.606240678 podStartE2EDuration="23.57072901s" totalImagesPullingTime="10.964488332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:55.570522588 +0000 UTC m=+23.721971956" watchObservedRunningTime="2026-07-23 14:52:55.57072901 +0000 UTC m=+23.722178384" Jul 23 14:52:55.571242 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.571215 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-11.ec2.internal" podStartSLOduration=22.571205414 podStartE2EDuration="22.571205414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:37.515162222 +0000 UTC m=+5.666611592" watchObservedRunningTime="2026-07-23 14:52:55.571205414 +0000 UTC m=+23.722654783" Jul 23 14:52:55.610893 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.610835 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8pjfm" podStartSLOduration=4.5725544 podStartE2EDuration="23.610815679s" totalImagesPullingTime="19.038261279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:55.6106298 +0000 UTC m=+23.762079174" watchObservedRunningTime="2026-07-23 14:52:55.610815679 +0000 UTC m=+23.762265049" Jul 23 14:52:55.611089 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.610996 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rfwvt" podStartSLOduration=4.514841033 podStartE2EDuration="23.610990042s" totalImagesPullingTime="19.096149009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:55.59479552 +0000 UTC m=+23.746244889" watchObservedRunningTime="2026-07-23 14:52:55.610990042 +0000 UTC m=+23.762439457" Jul 23 14:52:55.742145 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.742054 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ctr8r" podStartSLOduration=4.693289471 podStartE2EDuration="23.74203699s" totalImagesPullingTime="19.048747519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:55.687398484 +0000 UTC m=+23.838847856" watchObservedRunningTime="2026-07-23 14:52:55.74203699 +0000 UTC m=+23.893486359" Jul 23 14:52:55.789764 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:55.789729 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:56.008762 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.008735 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 14:52:56.261561 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.261396 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T14:52:56.008755623Z","UUID":"245bae4a-73f8-4ece-9a5c-47de2bfbaab7","Handler":null,"Name":"","Endpoint":""} Jul 23 14:52:56.263076 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.263055 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 14:52:56.263205 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.263099 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 14:52:56.287762 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.287734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:56.287953 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.287734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:56.287953 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:56.287882 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:56.288065 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:56.287953 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:56.351473 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.351434 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:56.352051 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.352036 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:56.372123 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.372076 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mfddd" podStartSLOduration=5.315675019 podStartE2EDuration="24.372052405s" totalImagesPullingTime="19.056377386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:55.743666729 +0000 UTC m=+23.895116100" watchObservedRunningTime="2026-07-23 14:52:56.372052405 +0000 UTC m=+24.523501819" Jul 23 14:52:56.568976 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.568899 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" event={"ID":"3be5364f-8f5b-48a1-bcd8-cecedcf0a176","Type":"ContainerStarted","Data":"25345df2010d5c855d525fa8d84be382729d6209aebfdb3fd1ac42830c3cd233"} Jul 23 14:52:56.570776 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.570697 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6c6ts" event={"ID":"15fd95ef-9ffd-4eaf-a1e0-8463fc986e43","Type":"ContainerStarted","Data":"d3d32e441bdbcc26556bcca83f7dea61a3e2e515f13d63b65460e60ac484ceae"} Jul 23 14:52:56.572448 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.572418 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nqjm5" Jul 23 14:52:56.588652 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:56.588600 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6c6ts" podStartSLOduration=5.549278135 podStartE2EDuration="24.588578816s" totalImagesPullingTime="19.039300681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:56.588570654 +0000 UTC m=+24.740020023" watchObservedRunningTime="2026-07-23 14:52:56.588578816 +0000 UTC m=+24.740028167" Jul 23 14:52:57.287567 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:57.287536 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:57.287725 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:57.287635 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:52:57.575012 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:57.574979 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" event={"ID":"3be5364f-8f5b-48a1-bcd8-cecedcf0a176","Type":"ContainerStarted","Data":"1a1703f3f5b0181a25762c9962840ab54ae00b2f38490e17734edec120f156f3"} Jul 23 14:52:57.579093 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:57.579056 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"3990329dcd28cba84a34edba67408ec4ec15f843cc3ab183437102dd4cf6beca"} Jul 23 14:52:58.291103 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:58.291074 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:52:58.291237 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:58.291174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:52:58.291237 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:58.291200 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:52:58.291338 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:58.291244 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:52:59.287790 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:52:59.287756 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:52:59.288206 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:52:59.287874 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:53:00.291158 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.290983 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:00.291776 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.291032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:00.291776 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:00.291243 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:53:00.291776 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:00.291301 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:53:00.587487 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.587410 2582 generic.go:356] "Generic (PLEG): container finished" podID="f5a883e7-68b9-4878-af6d-68a66b399ce3" containerID="784f7d4e3c6a04beec3cb0375d5bf0ca53837482673f6f59aeb7db8f7a3a92ef" exitCode=0 Jul 23 14:53:00.587487 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.587466 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerDied","Data":"784f7d4e3c6a04beec3cb0375d5bf0ca53837482673f6f59aeb7db8f7a3a92ef"} Jul 23 14:53:00.591710 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.591473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" event={"ID":"0181f318-c751-4ab6-bac0-cae5f18f87f0","Type":"ContainerStarted","Data":"8894002398d7e912944418d80555d99773403db3e734e7fd0aa09add3a6959b9"} Jul 23 14:53:00.591812 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.591780 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:53:00.591937 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.591836 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:53:00.591937 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.591884 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:53:00.607386 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.607330 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:53:00.607825 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.607749 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:53:00.610344 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.610308 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzg7w" podStartSLOduration=7.298200527 podStartE2EDuration="28.610295908s" totalImagesPullingTime="21.312095381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:52:57.624132798 +0000 UTC m=+25.775582168" watchObservedRunningTime="2026-07-23 14:53:00.610295908 +0000 UTC m=+28.761745274" Jul 23 14:53:00.636744 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:00.636679 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" podStartSLOduration=9.532081781 podStartE2EDuration="28.636658121s" totalImagesPullingTime="19.10457634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:53:00.634076859 +0000 UTC m=+28.785526248" watchObservedRunningTime="2026-07-23 14:53:00.636658121 +0000 UTC m=+28.788107489" Jul 23 14:53:01.287767 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.287736 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:01.288000 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:01.287843 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:53:01.890192 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.890162 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f2fgw"] Jul 23 14:53:01.890606 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.890280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:01.890606 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:01.890380 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:53:01.893371 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.893348 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xjwws"] Jul 23 14:53:01.893483 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.893445 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:01.893536 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:01.893520 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:53:01.896743 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.896719 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6nf8"] Jul 23 14:53:01.896850 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:01.896815 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:01.896904 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:01.896890 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:53:02.597770 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:02.597736 2582 generic.go:356] "Generic (PLEG): container finished" podID="f5a883e7-68b9-4878-af6d-68a66b399ce3" containerID="90ff2c31e67a48306120065e28acecf14e4dc7ee996d5199566a70ae5d87475e" exitCode=0 Jul 23 14:53:02.598004 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:02.597830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerDied","Data":"90ff2c31e67a48306120065e28acecf14e4dc7ee996d5199566a70ae5d87475e"} Jul 23 14:53:03.287872 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:03.287838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:03.288300 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:03.287904 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:03.288300 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:03.287913 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:03.288300 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:03.287995 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:53:03.288300 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:03.288073 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:53:03.288300 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:03.288154 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:53:04.604283 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:04.604234 2582 generic.go:356] "Generic (PLEG): container finished" podID="f5a883e7-68b9-4878-af6d-68a66b399ce3" containerID="9e144303516691d74271a7313038583d028ff6aad694afc55f767450402b669f" exitCode=0 Jul 23 14:53:04.604701 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:04.604292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerDied","Data":"9e144303516691d74271a7313038583d028ff6aad694afc55f767450402b669f"} Jul 23 14:53:05.288351 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:05.288143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:05.288514 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:05.288171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:05.288514 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:05.288451 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:53:05.288514 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:05.288182 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:05.288612 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:05.288585 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:53:05.288612 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:05.288495 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:53:06.066406 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:06.066373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:06.066847 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:06.066438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:06.066847 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.066533 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:53:06.066847 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.066535 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 14:53:06.066847 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.066585 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:53:38.066570524 +0000 UTC m=+66.218019871 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 14:53:06.066847 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.066598 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret podName:b8ac723b-5a09-485f-b5e9-faf6eef37039 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:38.066592055 +0000 UTC m=+66.218041401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret") pod "global-pull-secret-syncer-f2fgw" (UID: "b8ac723b-5a09-485f-b5e9-faf6eef37039") : object "kube-system"/"original-pull-secret" not registered Jul 23 14:53:06.167680 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:06.167643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:06.167841 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.167806 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 14:53:06.167841 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.167828 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 14:53:06.167841 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.167842 2582 projected.go:196] Error preparing data for projected volume kube-api-access-8q4hc for pod openshift-network-diagnostics/network-check-target-c6nf8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:53:06.167956 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:06.167902 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc podName:4d1abb73-8736-4de9-a115-5cce21a86a3a nodeName:}" failed. No retries permitted until 2026-07-23 14:53:38.167884034 +0000 UTC m=+66.319333397 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8q4hc" (UniqueName: "kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc") pod "network-check-target-c6nf8" (UID: "4d1abb73-8736-4de9-a115-5cce21a86a3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 14:53:07.288492 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:07.288456 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:07.288492 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:07.288478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:07.288492 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:07.288480 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:07.288949 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:07.288576 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f2fgw" podUID="b8ac723b-5a09-485f-b5e9-faf6eef37039" Jul 23 14:53:07.288949 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:07.288660 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:53:07.288949 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:07.288747 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6nf8" podUID="4d1abb73-8736-4de9-a115-5cce21a86a3a" Jul 23 14:53:08.184713 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.184684 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-11.ec2.internal" event="NodeReady" Jul 23 14:53:08.184891 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.184827 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 14:53:08.239156 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.239084 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dc6cc5c64-6rhfp"] Jul 23 14:53:08.242052 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.242025 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8"] Jul 23 14:53:08.242183 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.242156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.243880 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.243860 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.248804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.248747 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 14:53:08.251317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.249006 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 14:53:08.251317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.249028 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b8t5d\"" Jul 23 14:53:08.251317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.249319 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 14:53:08.251317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.249365 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 14:53:08.251317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.249742 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9zwgm\"" Jul 23 14:53:08.251317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.249792 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 14:53:08.256868 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.256845 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 14:53:08.258520 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.258499 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8"] Jul 23 14:53:08.261998 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.261943 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dc6cc5c64-6rhfp"] Jul 23 14:53:08.264549 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.264530 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ld8p4"] Jul 23 14:53:08.266779 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.266754 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.271720 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.271704 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zzbs4\"" Jul 23 14:53:08.271720 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.271717 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 14:53:08.271996 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.271975 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 14:53:08.272400 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.272384 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-67wsc"] Jul 23 14:53:08.276326 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.275393 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.278524 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.278504 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hrh8d\"" Jul 23 14:53:08.279136 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.279118 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 14:53:08.279240 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.279163 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 14:53:08.279240 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.279225 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 14:53:08.285297 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.285278 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-67wsc"] Jul 23 14:53:08.288278 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.288238 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ld8p4"] Jul 23 14:53:08.384382 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384347 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-image-registry-private-configuration\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384382 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a4c2533-1907-4fdc-aa02-4dcb7d526917-ca-trust-extracted\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384410 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-bound-sa-token\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfgs6\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-kube-api-access-gfgs6\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384504 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-trusted-ca\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384555 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b3ffade-f266-426b-be05-2e4ee4eb13db-tmp-dir\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384577 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxfqh\" (UniqueName: \"kubernetes.io/projected/9b3ffade-f266-426b-be05-2e4ee4eb13db-kube-api-access-nxfqh\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384602 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgwqb\" (UniqueName: \"kubernetes.io/projected/c15231de-2b65-4e56-8295-96311beed997-kube-api-access-qgwqb\") pod \"ingress-canary-67wsc\" (UID: \"c15231de-2b65-4e56-8295-96311beed997\") " pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384655 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-certificates\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3ffade-f266-426b-be05-2e4ee4eb13db-config-volume\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384709 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08c05031-e76c-49da-94f0-a6329db3d717-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384736 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384833 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-installation-pull-secrets\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.384859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c15231de-2b65-4e56-8295-96311beed997-cert\") pod \"ingress-canary-67wsc\" (UID: \"c15231de-2b65-4e56-8295-96311beed997\") " pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.385331 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.384926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485634 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-installation-pull-secrets\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485634 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c15231de-2b65-4e56-8295-96311beed997-cert\") pod \"ingress-canary-67wsc\" (UID: \"c15231de-2b65-4e56-8295-96311beed997\") " pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485702 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485735 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-image-registry-private-configuration\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485758 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a4c2533-1907-4fdc-aa02-4dcb7d526917-ca-trust-extracted\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-bound-sa-token\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfgs6\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-kube-api-access-gfgs6\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-trusted-ca\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.485876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485863 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.485875 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485885 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b3ffade-f266-426b-be05-2e4ee4eb13db-tmp-dir\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.485895 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485909 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxfqh\" (UniqueName: \"kubernetes.io/projected/9b3ffade-f266-426b-be05-2e4ee4eb13db-kube-api-access-nxfqh\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgwqb\" (UniqueName: \"kubernetes.io/projected/c15231de-2b65-4e56-8295-96311beed997-kube-api-access-qgwqb\") pod \"ingress-canary-67wsc\" (UID: \"c15231de-2b65-4e56-8295-96311beed997\") " pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.485955 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:08.985939715 +0000 UTC m=+37.137389061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.485985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-certificates\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.486005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3ffade-f266-426b-be05-2e4ee4eb13db-config-volume\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.486022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08c05031-e76c-49da-94f0-a6329db3d717-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.486048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.486127 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:08.486169 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.486153 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:08.986145047 +0000 UTC m=+37.137594394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:08.486740 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.486622 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a4c2533-1907-4fdc-aa02-4dcb7d526917-ca-trust-extracted\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.486740 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.486648 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b3ffade-f266-426b-be05-2e4ee4eb13db-tmp-dir\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.486829 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.486757 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:08.486829 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.486808 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:53:08.986793476 +0000 UTC m=+37.138242825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:08.486935 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.486913 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3ffade-f266-426b-be05-2e4ee4eb13db-config-volume\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.487154 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.487050 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/08c05031-e76c-49da-94f0-a6329db3d717-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.487281 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.487228 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-trusted-ca\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.487355 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.487309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-certificates\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.490733 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.490671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-image-registry-private-configuration\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.490733 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.490680 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-installation-pull-secrets\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.490733 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.490707 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c15231de-2b65-4e56-8295-96311beed997-cert\") pod \"ingress-canary-67wsc\" (UID: \"c15231de-2b65-4e56-8295-96311beed997\") " pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.497038 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.497018 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-bound-sa-token\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.497365 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.497334 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxfqh\" (UniqueName: \"kubernetes.io/projected/9b3ffade-f266-426b-be05-2e4ee4eb13db-kube-api-access-nxfqh\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.497438 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.497389 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgwqb\" (UniqueName: \"kubernetes.io/projected/c15231de-2b65-4e56-8295-96311beed997-kube-api-access-qgwqb\") pod \"ingress-canary-67wsc\" (UID: \"c15231de-2b65-4e56-8295-96311beed997\") " pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.498814 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.498793 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfgs6\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-kube-api-access-gfgs6\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.589400 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.589363 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-67wsc" Jul 23 14:53:08.989679 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.989514 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:08.989813 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.989728 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:08.989813 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.989658 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:08.989912 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.989818 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:53:09.989796692 +0000 UTC m=+38.141246042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:08.989912 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.989873 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:08.989912 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:08.989881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:08.990061 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.989928 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:09.989910978 +0000 UTC m=+38.141360330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:08.990061 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.989995 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:08.990061 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.990008 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:08.990061 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:08.990053 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:09.990038854 +0000 UTC m=+38.141488203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:09.024747 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.024721 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-67wsc"] Jul 23 14:53:09.027705 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:53:09.027680 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15231de_2b65_4e56_8295_96311beed997.slice/crio-ac1919056823c62a45d003f76ed5b17f52578965b1a119dd8f25b7bdbc308ddd WatchSource:0}: Error finding container ac1919056823c62a45d003f76ed5b17f52578965b1a119dd8f25b7bdbc308ddd: Status 404 returned error can't find the container with id ac1919056823c62a45d003f76ed5b17f52578965b1a119dd8f25b7bdbc308ddd Jul 23 14:53:09.287789 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.287756 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:09.287789 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.287787 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:09.288029 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.288009 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:09.290840 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.290822 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 14:53:09.292144 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.292113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 14:53:09.292242 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.292119 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 14:53:09.292242 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.292221 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-blqvw\"" Jul 23 14:53:09.292376 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.292119 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 14:53:09.292376 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.292122 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-brw86\"" Jul 23 14:53:09.620552 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.620475 2582 generic.go:356] "Generic (PLEG): container finished" podID="f5a883e7-68b9-4878-af6d-68a66b399ce3" containerID="8915c598d848a44579d12e822fa67a9f02cc593bee709f3ea75e7eb3238698ea" exitCode=0 Jul 23 14:53:09.621336 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.620549 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerDied","Data":"8915c598d848a44579d12e822fa67a9f02cc593bee709f3ea75e7eb3238698ea"} Jul 23 14:53:09.621636 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.621618 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-67wsc" event={"ID":"c15231de-2b65-4e56-8295-96311beed997","Type":"ContainerStarted","Data":"ac1919056823c62a45d003f76ed5b17f52578965b1a119dd8f25b7bdbc308ddd"} Jul 23 14:53:09.997895 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.997856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:09.998067 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.997914 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:09.998067 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:09.998013 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:09.998067 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998047 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:09.998218 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998112 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:53:11.998097706 +0000 UTC m=+40.149547053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:09.998218 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998152 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:09.998218 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998172 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:09.998472 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998226 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:11.998208527 +0000 UTC m=+40.149657891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:09.998472 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998048 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:09.998472 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:09.998289 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:11.998277816 +0000 UTC m=+40.149727193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:10.628214 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:10.628173 2582 generic.go:356] "Generic (PLEG): container finished" podID="f5a883e7-68b9-4878-af6d-68a66b399ce3" containerID="f3a736db0b973b31832218f2ba12ec3ee07713850fa23ae684d3c00173c47e39" exitCode=0 Jul 23 14:53:10.628608 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:10.628244 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerDied","Data":"f3a736db0b973b31832218f2ba12ec3ee07713850fa23ae684d3c00173c47e39"} Jul 23 14:53:11.633790 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:11.633758 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" event={"ID":"f5a883e7-68b9-4878-af6d-68a66b399ce3","Type":"ContainerStarted","Data":"7f459a1b6da3e2e2ac0056f300407a4c0b2b0063ea027966e8996c40635c66f9"} Jul 23 14:53:11.662797 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:11.662748 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rqkgc" podStartSLOduration=9.508695262 podStartE2EDuration="39.662733018s" totalImagesPullingTime="30.154037756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:53:11.661122486 +0000 UTC m=+39.812571855" watchObservedRunningTime="2026-07-23 14:53:11.662733018 +0000 UTC m=+39.814182386" Jul 23 14:53:12.016317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:12.016287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:12.016459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:12.016328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:12.016459 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016434 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:12.016459 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016436 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:12.016618 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:12.016489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:12.016618 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016499 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:53:16.016481832 +0000 UTC m=+44.167931193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:12.016618 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016547 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:16.016535646 +0000 UTC m=+44.167984999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:12.016618 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016566 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:12.016618 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016576 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:12.016618 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:12.016609 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:16.016598782 +0000 UTC m=+44.168048150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:12.637085 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:12.637048 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-67wsc" event={"ID":"c15231de-2b65-4e56-8295-96311beed997","Type":"ContainerStarted","Data":"6e5e67ed6fb32c9b8d349f893bf8362daeaf61073b8a4d01a652ae443fb38af3"} Jul 23 14:53:12.669388 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:12.669334 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-67wsc" podStartSLOduration=1.8034832079999998 podStartE2EDuration="4.669318736s" totalImagesPullingTime="2.865835528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:53:12.668904078 +0000 UTC m=+40.820353446" watchObservedRunningTime="2026-07-23 14:53:12.669318736 +0000 UTC m=+40.820768105" Jul 23 14:53:16.049691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:16.049644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:16.049691 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:16.049691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049790 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049794 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049844 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:24.049829217 +0000 UTC m=+52.201278564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:16.049864 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049927 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:53:24.049908727 +0000 UTC m=+52.201358078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049933 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049947 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:16.050113 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:16.049973 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:24.049966243 +0000 UTC m=+52.201415589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:24.112175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:24.112132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:24.112175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:24.112184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:24.112211 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112334 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112340 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112375 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112395 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:53:40.112379811 +0000 UTC m=+68.263829159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112441 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:40.112422167 +0000 UTC m=+68.263871534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112337 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:24.112749 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:24.112493 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:53:40.112481138 +0000 UTC m=+68.263930489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:32.608400 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:32.608371 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2svvx" Jul 23 14:53:38.116785 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.116733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:53:38.116785 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.116793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:38.120056 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.120036 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 14:53:38.120203 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.120185 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 14:53:38.127522 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:38.127503 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 23 14:53:38.127594 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:38.127558 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:54:42.127542783 +0000 UTC m=+130.278992130 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : secret "metrics-daemon-secret" not found Jul 23 14:53:38.131153 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.131132 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b8ac723b-5a09-485f-b5e9-faf6eef37039-original-pull-secret\") pod \"global-pull-secret-syncer-f2fgw\" (UID: \"b8ac723b-5a09-485f-b5e9-faf6eef37039\") " pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:38.218138 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.218087 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:38.221673 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.221655 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 14:53:38.232174 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.232157 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 14:53:38.242025 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.241999 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q4hc\" (UniqueName: \"kubernetes.io/projected/4d1abb73-8736-4de9-a115-5cce21a86a3a-kube-api-access-8q4hc\") pod \"network-check-target-c6nf8\" (UID: \"4d1abb73-8736-4de9-a115-5cce21a86a3a\") " pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:38.400214 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.400191 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-blqvw\"" Jul 23 14:53:38.407966 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.407943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:38.408081 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.408014 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f2fgw" Jul 23 14:53:38.541321 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.541287 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6nf8"] Jul 23 14:53:38.545292 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:53:38.545245 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1abb73_8736_4de9_a115_5cce21a86a3a.slice/crio-f370d807278d1c82cba4581cbb347ce76e903a799e3c58e60e8bcac01240ed34 WatchSource:0}: Error finding container f370d807278d1c82cba4581cbb347ce76e903a799e3c58e60e8bcac01240ed34: Status 404 returned error can't find the container with id f370d807278d1c82cba4581cbb347ce76e903a799e3c58e60e8bcac01240ed34 Jul 23 14:53:38.554536 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.554515 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f2fgw"] Jul 23 14:53:38.557470 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:53:38.557448 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ac723b_5a09_485f_b5e9_faf6eef37039.slice/crio-0e296c98e582e6b94da96e740936f97514d8692e8926121b2284b19a687b8fd8 WatchSource:0}: Error finding container 0e296c98e582e6b94da96e740936f97514d8692e8926121b2284b19a687b8fd8: Status 404 returned error can't find the container with id 0e296c98e582e6b94da96e740936f97514d8692e8926121b2284b19a687b8fd8 Jul 23 14:53:38.694949 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.694866 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f2fgw" event={"ID":"b8ac723b-5a09-485f-b5e9-faf6eef37039","Type":"ContainerStarted","Data":"0e296c98e582e6b94da96e740936f97514d8692e8926121b2284b19a687b8fd8"} Jul 23 14:53:38.696085 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:38.696061 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6nf8" event={"ID":"4d1abb73-8736-4de9-a115-5cce21a86a3a","Type":"ContainerStarted","Data":"f370d807278d1c82cba4581cbb347ce76e903a799e3c58e60e8bcac01240ed34"} Jul 23 14:53:40.131618 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:40.131574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:40.131652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:40.131692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131752 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131775 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131808 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131816 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131838 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:54:12.131818683 +0000 UTC m=+100.283268033 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131861 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:54:12.131848578 +0000 UTC m=+100.283297925 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:53:40.132059 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:53:40.131874 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:54:12.131868136 +0000 UTC m=+100.283317483 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:53:43.711133 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:43.711095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f2fgw" event={"ID":"b8ac723b-5a09-485f-b5e9-faf6eef37039","Type":"ContainerStarted","Data":"4f33ee1367adf6b1d9baea2a40a1dffe691a3e147b7e85875a53a68a4090f43f"} Jul 23 14:53:43.712523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:43.712501 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6nf8" event={"ID":"4d1abb73-8736-4de9-a115-5cce21a86a3a","Type":"ContainerStarted","Data":"a50c2c45d5e128113257c3fad424f3878c79325f3bd2229c461972b20296b2f1"} Jul 23 14:53:43.712647 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:43.712627 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:53:43.727762 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:43.727720 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f2fgw" podStartSLOduration=67.058154725 podStartE2EDuration="1m11.727707274s" totalImagesPullingTime="4.669552549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:53:43.727208186 +0000 UTC m=+71.878657556" watchObservedRunningTime="2026-07-23 14:53:43.727707274 +0000 UTC m=+71.879156644" Jul 23 14:53:43.741378 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:53:43.741339 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c6nf8" podStartSLOduration=67.06300819 podStartE2EDuration="1m11.741327839s" totalImagesPullingTime="4.678319649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:53:43.740863499 +0000 UTC m=+71.892312867" watchObservedRunningTime="2026-07-23 14:53:43.741327839 +0000 UTC m=+71.892777244" Jul 23 14:54:12.160801 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:54:12.160766 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:54:12.160823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:54:12.160849 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.160932 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.160942 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.160952 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.160953 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.161007 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:55:16.16099399 +0000 UTC m=+164.312443337 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.161019 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:55:16.161013544 +0000 UTC m=+164.312462891 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:54:12.161350 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:12.161028 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:55:16.161023345 +0000 UTC m=+164.312472693 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:54:14.717075 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:54:14.717042 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c6nf8" Jul 23 14:54:42.176858 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:54:42.176799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:54:42.177427 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:42.176960 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 23 14:54:42.177427 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:54:42.177025 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:56:44.177008849 +0000 UTC m=+252.328458196 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : secret "metrics-daemon-secret" not found Jul 23 14:55:11.257027 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:11.256977 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" Jul 23 14:55:11.264144 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:11.264119 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" podUID="08c05031-e76c-49da-94f0-a6329db3d717" Jul 23 14:55:11.279348 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:11.279327 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ld8p4" podUID="9b3ffade-f266-426b-be05-2e4ee4eb13db" Jul 23 14:55:11.913729 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:11.913703 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:55:11.913729 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:11.913722 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld8p4" Jul 23 14:55:11.913926 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:11.913899 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:55:12.302889 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:12.302801 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:55:16.225491 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:16.225453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:55:16.225491 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:16.225493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:16.225575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225612 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225662 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert podName:08c05031-e76c-49da-94f0-a6329db3d717 nodeName:}" failed. No retries permitted until 2026-07-23 14:57:18.225646915 +0000 UTC m=+286.377096261 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9lsg8" (UID: "08c05031-e76c-49da-94f0-a6329db3d717") : secret "networking-console-plugin-cert" not found Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225721 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225736 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dc6cc5c64-6rhfp: secret "image-registry-tls" not found Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225736 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225791 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls podName:9b3ffade-f266-426b-be05-2e4ee4eb13db nodeName:}" failed. No retries permitted until 2026-07-23 14:57:18.225778841 +0000 UTC m=+286.377228191 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls") pod "dns-default-ld8p4" (UID: "9b3ffade-f266-426b-be05-2e4ee4eb13db") : secret "dns-default-metrics-tls" not found Jul 23 14:55:16.226058 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:55:16.225809 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls podName:1a4c2533-1907-4fdc-aa02-4dcb7d526917 nodeName:}" failed. No retries permitted until 2026-07-23 14:57:18.225799583 +0000 UTC m=+286.377248933 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls") pod "image-registry-7dc6cc5c64-6rhfp" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917") : secret "image-registry-tls" not found Jul 23 14:55:27.287797 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:55:27.287718 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:56:44.221012 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.220970 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29"] Jul 23 14:56:44.222779 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.222762 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94"] Jul 23 14:56:44.222903 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.222886 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" Jul 23 14:56:44.224439 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.224419 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ctddv"] Jul 23 14:56:44.224560 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.224521 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" Jul 23 14:56:44.225557 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.225539 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vfrw5\"" Jul 23 14:56:44.225778 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.225761 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 14:56:44.225859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.225779 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 14:56:44.226032 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.226016 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5"] Jul 23 14:56:44.226130 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.226117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.227022 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.227006 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-drr82\"" Jul 23 14:56:44.227417 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.227401 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.228776 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.228760 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 14:56:44.228853 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.228843 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 14:56:44.229303 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.229288 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 14:56:44.229815 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.229460 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 14:56:44.229815 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.229503 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-drqnr\"" Jul 23 14:56:44.230371 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.230354 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 14:56:44.230924 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.230904 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 14:56:44.232468 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.232446 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 14:56:44.232676 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.232655 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-68qlm\"" Jul 23 14:56:44.232762 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.232666 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 14:56:44.234993 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.234974 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29"] Jul 23 14:56:44.237095 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.237015 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 14:56:44.237206 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.237192 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94"] Jul 23 14:56:44.239441 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239424 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5"] Jul 23 14:56:44.239547 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/f006660f-7327-49bd-b503-9adf44fe4ff9-kube-api-access-d6hdv\") pod \"network-check-source-7bc5b6bf85-lrr94\" (UID: \"f006660f-7327-49bd-b503-9adf44fe4ff9\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" Jul 23 14:56:44.239547 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.239664 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239574 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-serving-cert\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.239664 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f90fd981-0d3a-4d04-adda-23df0f99ea87-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.239664 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztmmb\" (UniqueName: \"kubernetes.io/projected/8adfbf4b-b4e4-43cb-b2d0-72c75caa6939-kube-api-access-ztmmb\") pod \"volume-data-source-validator-78cf4c6f6-vts29\" (UID: \"8adfbf4b-b4e4-43cb-b2d0-72c75caa6939\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239704 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-trusted-ca\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.239714 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w7mt\" (UniqueName: \"kubernetes.io/projected/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-kube-api-access-5w7mt\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239737 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-config\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.239752 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm8rz\" (UniqueName: \"kubernetes.io/projected/f90fd981-0d3a-4d04-adda-23df0f99ea87-kube-api-access-nm8rz\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.239804 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.239774 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs podName:3d60ecea-1743-434c-8b1c-78b027daac9b nodeName:}" failed. No retries permitted until 2026-07-23 14:58:46.239756784 +0000 UTC m=+374.391206150 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs") pod "network-metrics-daemon-xjwws" (UID: "3d60ecea-1743-434c-8b1c-78b027daac9b") : secret "metrics-daemon-secret" not found Jul 23 14:56:44.242403 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.242382 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ctddv"] Jul 23 14:56:44.313633 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.313604 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd"] Jul 23 14:56:44.315616 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.315602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.317888 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.317861 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-85f8bf867b-ckpfm"] Jul 23 14:56:44.319695 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.319678 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.322527 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.322509 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 14:56:44.322527 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.322521 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 14:56:44.322653 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.322537 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 14:56:44.322840 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.322827 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 14:56:44.322881 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.322858 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 14:56:44.323163 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.323145 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4897h\"" Jul 23 14:56:44.323244 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.323148 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 14:56:44.323466 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.323448 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 14:56:44.323657 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.323631 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qnq5n\"" Jul 23 14:56:44.324812 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.324793 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 14:56:44.325328 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.325311 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 14:56:44.326316 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.326293 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 14:56:44.332327 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.332307 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd"] Jul 23 14:56:44.336410 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.336391 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85f8bf867b-ckpfm"] Jul 23 14:56:44.340666 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5acaf8bb-5254-4bba-97af-d0870a555d89-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.340756 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340676 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-trusted-ca\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.340756 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340694 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm8rz\" (UniqueName: \"kubernetes.io/projected/f90fd981-0d3a-4d04-adda-23df0f99ea87-kube-api-access-nm8rz\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.340756 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340711 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-config\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.340756 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5acaf8bb-5254-4bba-97af-d0870a555d89-config\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.340954 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340828 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.340954 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340851 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-serving-cert\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.340954 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340867 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-default-certificate\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.340954 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340893 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-stats-auth\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.340954 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f90fd981-0d3a-4d04-adda-23df0f99ea87-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.341194 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.340965 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztmmb\" (UniqueName: \"kubernetes.io/projected/8adfbf4b-b4e4-43cb-b2d0-72c75caa6939-kube-api-access-ztmmb\") pod \"volume-data-source-validator-78cf4c6f6-vts29\" (UID: \"8adfbf4b-b4e4-43cb-b2d0-72c75caa6939\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" Jul 23 14:56:44.341194 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg9dm\" (UniqueName: \"kubernetes.io/projected/5acaf8bb-5254-4bba-97af-d0870a555d89-kube-api-access-pg9dm\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.341194 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w7mt\" (UniqueName: \"kubernetes.io/projected/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-kube-api-access-5w7mt\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.341194 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341092 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.341194 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341138 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss6jb\" (UniqueName: \"kubernetes.io/projected/46c81778-15ff-474c-a0f3-10a6458b470a-kube-api-access-ss6jb\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.341194 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341172 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/f006660f-7327-49bd-b503-9adf44fe4ff9-kube-api-access-d6hdv\") pod \"network-check-source-7bc5b6bf85-lrr94\" (UID: \"f006660f-7327-49bd-b503-9adf44fe4ff9\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" Jul 23 14:56:44.341505 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.341505 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.341390 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:44.341505 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.341447 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls podName:f90fd981-0d3a-4d04-adda-23df0f99ea87 nodeName:}" failed. No retries permitted until 2026-07-23 14:56:44.841429594 +0000 UTC m=+252.992878956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-58sm5" (UID: "f90fd981-0d3a-4d04-adda-23df0f99ea87") : secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:44.341505 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341478 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-config\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.341719 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341621 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f90fd981-0d3a-4d04-adda-23df0f99ea87-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.341777 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.341762 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-trusted-ca\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.343211 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.343189 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-serving-cert\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.360882 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.360860 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm8rz\" (UniqueName: \"kubernetes.io/projected/f90fd981-0d3a-4d04-adda-23df0f99ea87-kube-api-access-nm8rz\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.361104 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.361085 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w7mt\" (UniqueName: \"kubernetes.io/projected/097ab5e1-b387-48d5-a9a5-c4a604bd98bc-kube-api-access-5w7mt\") pod \"console-operator-565b7b569d-ctddv\" (UID: \"097ab5e1-b387-48d5-a9a5-c4a604bd98bc\") " pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.361348 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.361330 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/f006660f-7327-49bd-b503-9adf44fe4ff9-kube-api-access-d6hdv\") pod \"network-check-source-7bc5b6bf85-lrr94\" (UID: \"f006660f-7327-49bd-b503-9adf44fe4ff9\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" Jul 23 14:56:44.361396 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.361380 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztmmb\" (UniqueName: \"kubernetes.io/projected/8adfbf4b-b4e4-43cb-b2d0-72c75caa6939-kube-api-access-ztmmb\") pod \"volume-data-source-validator-78cf4c6f6-vts29\" (UID: \"8adfbf4b-b4e4-43cb-b2d0-72c75caa6939\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" Jul 23 14:56:44.442090 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442053 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.442090 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-default-certificate\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-stats-auth\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442175 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg9dm\" (UniqueName: \"kubernetes.io/projected/5acaf8bb-5254-4bba-97af-d0870a555d89-kube-api-access-pg9dm\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.442204 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.442300 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:44.942280235 +0000 UTC m=+253.093729583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : secret "router-metrics-certs-default" not found Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.442324 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:44.942312726 +0000 UTC m=+253.093762088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : configmap references non-existent config key: service-ca.crt Jul 23 14:56:44.442383 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss6jb\" (UniqueName: \"kubernetes.io/projected/46c81778-15ff-474c-a0f3-10a6458b470a-kube-api-access-ss6jb\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.442805 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442430 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5acaf8bb-5254-4bba-97af-d0870a555d89-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.442805 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5acaf8bb-5254-4bba-97af-d0870a555d89-config\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.442990 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.442970 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5acaf8bb-5254-4bba-97af-d0870a555d89-config\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.444659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.444634 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5acaf8bb-5254-4bba-97af-d0870a555d89-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.444659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.444648 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-default-certificate\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.444790 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.444773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-stats-auth\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.454131 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.454110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg9dm\" (UniqueName: \"kubernetes.io/projected/5acaf8bb-5254-4bba-97af-d0870a555d89-kube-api-access-pg9dm\") pod \"service-ca-operator-5bcbc94c78-hmpxd\" (UID: \"5acaf8bb-5254-4bba-97af-d0870a555d89\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.454216 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.454169 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss6jb\" (UniqueName: \"kubernetes.io/projected/46c81778-15ff-474c-a0f3-10a6458b470a-kube-api-access-ss6jb\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.533040 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.532952 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" Jul 23 14:56:44.541937 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.541918 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" Jul 23 14:56:44.548665 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.548644 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:44.625454 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.625341 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" Jul 23 14:56:44.687789 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.687763 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29"] Jul 23 14:56:44.702197 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.702172 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94"] Jul 23 14:56:44.705569 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:56:44.705539 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf006660f_7327_49bd_b503_9adf44fe4ff9.slice/crio-20f11e02043b96d14f460af267996968b96b2bb50d292a856024a24e82e380bd WatchSource:0}: Error finding container 20f11e02043b96d14f460af267996968b96b2bb50d292a856024a24e82e380bd: Status 404 returned error can't find the container with id 20f11e02043b96d14f460af267996968b96b2bb50d292a856024a24e82e380bd Jul 23 14:56:44.762268 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.762219 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd"] Jul 23 14:56:44.764728 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:56:44.764702 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5acaf8bb_5254_4bba_97af_d0870a555d89.slice/crio-bf5fde756ebb2d0c561aef261fb0b8488320e0c58a613b2f39360fa75ac0a573 WatchSource:0}: Error finding container bf5fde756ebb2d0c561aef261fb0b8488320e0c58a613b2f39360fa75ac0a573: Status 404 returned error can't find the container with id bf5fde756ebb2d0c561aef261fb0b8488320e0c58a613b2f39360fa75ac0a573 Jul 23 14:56:44.846386 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.846307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:44.846520 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.846451 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:44.846520 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.846518 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls podName:f90fd981-0d3a-4d04-adda-23df0f99ea87 nodeName:}" failed. No retries permitted until 2026-07-23 14:56:45.846503042 +0000 UTC m=+253.997952388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-58sm5" (UID: "f90fd981-0d3a-4d04-adda-23df0f99ea87") : secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:44.904814 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.904785 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ctddv"] Jul 23 14:56:44.907766 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:56:44.907740 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097ab5e1_b387_48d5_a9a5_c4a604bd98bc.slice/crio-79866e3dfd9181679c0a12cf9e38ba20bc11d994060b34903e4e5d77aa86fd08 WatchSource:0}: Error finding container 79866e3dfd9181679c0a12cf9e38ba20bc11d994060b34903e4e5d77aa86fd08: Status 404 returned error can't find the container with id 79866e3dfd9181679c0a12cf9e38ba20bc11d994060b34903e4e5d77aa86fd08 Jul 23 14:56:44.946757 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.946734 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.946865 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:44.946829 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:44.946917 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.946876 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 14:56:44.946959 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.946939 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:45.946922609 +0000 UTC m=+254.098371962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : secret "router-metrics-certs-default" not found Jul 23 14:56:44.947002 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:44.946982 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:45.946965427 +0000 UTC m=+254.098414788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : configmap references non-existent config key: service-ca.crt Jul 23 14:56:45.136737 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.136660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" event={"ID":"f006660f-7327-49bd-b503-9adf44fe4ff9","Type":"ContainerStarted","Data":"6191053c450ecb66a370a3a88fde27556687c6d8cb3b3d5436941110d50839e5"} Jul 23 14:56:45.136737 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.136711 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" event={"ID":"f006660f-7327-49bd-b503-9adf44fe4ff9","Type":"ContainerStarted","Data":"20f11e02043b96d14f460af267996968b96b2bb50d292a856024a24e82e380bd"} Jul 23 14:56:45.137827 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.137796 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" event={"ID":"097ab5e1-b387-48d5-a9a5-c4a604bd98bc","Type":"ContainerStarted","Data":"79866e3dfd9181679c0a12cf9e38ba20bc11d994060b34903e4e5d77aa86fd08"} Jul 23 14:56:45.138905 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.138884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" event={"ID":"5acaf8bb-5254-4bba-97af-d0870a555d89","Type":"ContainerStarted","Data":"bf5fde756ebb2d0c561aef261fb0b8488320e0c58a613b2f39360fa75ac0a573"} Jul 23 14:56:45.139890 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.139871 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" event={"ID":"8adfbf4b-b4e4-43cb-b2d0-72c75caa6939","Type":"ContainerStarted","Data":"fc4790f355956b126237aecc9623453e42d924c71cde3414cfbfc16b4c8139ec"} Jul 23 14:56:45.853373 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.853333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:45.853823 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:45.853497 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:45.853823 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:45.853577 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls podName:f90fd981-0d3a-4d04-adda-23df0f99ea87 nodeName:}" failed. No retries permitted until 2026-07-23 14:56:47.853555513 +0000 UTC m=+256.005004874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-58sm5" (UID: "f90fd981-0d3a-4d04-adda-23df0f99ea87") : secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:45.954788 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.954743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:45.954987 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:45.954859 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:45.954987 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:45.954922 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 14:56:45.954987 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:45.954979 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:47.954959312 +0000 UTC m=+256.106408675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : configmap references non-existent config key: service-ca.crt Jul 23 14:56:45.955159 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:45.955004 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:47.954993594 +0000 UTC m=+256.106442946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : secret "router-metrics-certs-default" not found Jul 23 14:56:47.151402 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:47.151364 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" event={"ID":"8adfbf4b-b4e4-43cb-b2d0-72c75caa6939","Type":"ContainerStarted","Data":"e3ab5af7adf42f48ea9cdb2ae690dded756a98d45f84c83f957fd7104aaa4823"} Jul 23 14:56:47.169145 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:47.168981 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-lrr94" podStartSLOduration=3.168961432 podStartE2EDuration="3.168961432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:56:45.152589122 +0000 UTC m=+253.304038491" watchObservedRunningTime="2026-07-23 14:56:47.168961432 +0000 UTC m=+255.320410802" Jul 23 14:56:47.169511 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:47.169469 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vts29" podStartSLOduration=1.683755181 podStartE2EDuration="3.169459165s" totalImagesPullingTime="1.485703984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:56:47.168662313 +0000 UTC m=+255.320111674" watchObservedRunningTime="2026-07-23 14:56:47.169459165 +0000 UTC m=+255.320908534" Jul 23 14:56:47.871359 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:47.871239 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:47.871509 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:47.871411 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:47.871509 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:47.871489 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls podName:f90fd981-0d3a-4d04-adda-23df0f99ea87 nodeName:}" failed. No retries permitted until 2026-07-23 14:56:51.871468543 +0000 UTC m=+260.022917893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-58sm5" (UID: "f90fd981-0d3a-4d04-adda-23df0f99ea87") : secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:47.972578 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:47.972537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:47.972761 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:47.972632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:47.972761 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:47.972700 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 14:56:47.972761 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:47.972757 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:51.972743256 +0000 UTC m=+260.124192603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : secret "router-metrics-certs-default" not found Jul 23 14:56:47.972862 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:47.972772 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:51.972765823 +0000 UTC m=+260.124215170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : configmap references non-existent config key: service-ca.crt Jul 23 14:56:48.091427 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.091386 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-96lf4"] Jul 23 14:56:48.093723 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.093707 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.097133 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.097112 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 14:56:48.097539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.097525 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 14:56:48.098528 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.098502 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 14:56:48.098620 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.098499 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pmdlz\"" Jul 23 14:56:48.098620 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.098544 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 14:56:48.111985 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.109563 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-96lf4"] Jul 23 14:56:48.156609 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.156574 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/0.log" Jul 23 14:56:48.157080 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.156628 2582 generic.go:356] "Generic (PLEG): container finished" podID="097ab5e1-b387-48d5-a9a5-c4a604bd98bc" containerID="8dc0504e9872400e98a856fd99a76e2968c5c38d561340ffd8a0af7c68072c85" exitCode=255 Jul 23 14:56:48.157080 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.157019 2582 scope.go:117] "RemoveContainer" containerID="8dc0504e9872400e98a856fd99a76e2968c5c38d561340ffd8a0af7c68072c85" Jul 23 14:56:48.157725 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.157233 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" event={"ID":"097ab5e1-b387-48d5-a9a5-c4a604bd98bc","Type":"ContainerDied","Data":"8dc0504e9872400e98a856fd99a76e2968c5c38d561340ffd8a0af7c68072c85"} Jul 23 14:56:48.159677 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.159650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" event={"ID":"5acaf8bb-5254-4bba-97af-d0870a555d89","Type":"ContainerStarted","Data":"6fd633d92c9817c867178cae9e4b7a1cfa88a692d2a98ef4b15785c109ab72dd"} Jul 23 14:56:48.175612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.175566 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.175752 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.175699 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.175821 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.175750 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b959cf26-685b-48c2-9b92-8969331ab09a-data-volume\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.175821 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.175811 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br4cl\" (UniqueName: \"kubernetes.io/projected/b959cf26-685b-48c2-9b92-8969331ab09a-kube-api-access-br4cl\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.175929 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.175891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b959cf26-685b-48c2-9b92-8969331ab09a-crio-socket\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.217215 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.217177 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" podStartSLOduration=1.44672135 podStartE2EDuration="4.217161239s" totalImagesPullingTime="2.770439889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:56:48.216950228 +0000 UTC m=+256.368399608" watchObservedRunningTime="2026-07-23 14:56:48.217161239 +0000 UTC m=+256.368610608" Jul 23 14:56:48.277387 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b959cf26-685b-48c2-9b92-8969331ab09a-crio-socket\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277499 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b959cf26-685b-48c2-9b92-8969331ab09a-crio-socket\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277523 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277518 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277703 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b959cf26-685b-48c2-9b92-8969331ab09a-data-volume\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277703 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.277661 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 14:56:48.277872 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.277734 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:48.77771245 +0000 UTC m=+256.929161821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : secret "insights-runtime-extractor-tls" not found Jul 23 14:56:48.277872 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277662 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br4cl\" (UniqueName: \"kubernetes.io/projected/b959cf26-685b-48c2-9b92-8969331ab09a-kube-api-access-br4cl\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277976 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.277915 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b959cf26-685b-48c2-9b92-8969331ab09a-data-volume\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.277976 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.277925 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 14:56:48.277976 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.277974 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:48.777958011 +0000 UTC m=+256.929407373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : configmap "kube-rbac-proxy" not found Jul 23 14:56:48.286567 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.286540 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br4cl\" (UniqueName: \"kubernetes.io/projected/b959cf26-685b-48c2-9b92-8969331ab09a-kube-api-access-br4cl\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.781497 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.781466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.781699 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:48.781511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:48.781699 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.781590 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 14:56:48.781699 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.781624 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 14:56:48.781699 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.781668 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:49.781642006 +0000 UTC m=+257.933091355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : configmap "kube-rbac-proxy" not found Jul 23 14:56:48.781699 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:48.781684 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:49.781678397 +0000 UTC m=+257.933127744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : secret "insights-runtime-extractor-tls" not found Jul 23 14:56:49.049201 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.049125 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q"] Jul 23 14:56:49.051189 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.051173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" Jul 23 14:56:49.054482 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.054459 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6nzb8\"" Jul 23 14:56:49.054590 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.054464 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 23 14:56:49.055786 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.055768 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 23 14:56:49.073448 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.073429 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q"] Jul 23 14:56:49.084547 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.084527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqcpv\" (UniqueName: \"kubernetes.io/projected/5f04272a-404d-4370-96bb-019a750f3b49-kube-api-access-pqcpv\") pod \"migrator-7cc9c7ffd7-zk47q\" (UID: \"5f04272a-404d-4370-96bb-019a750f3b49\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" Jul 23 14:56:49.163938 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.163914 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 14:56:49.164355 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.164343 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/0.log" Jul 23 14:56:49.164412 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.164374 2582 generic.go:356] "Generic (PLEG): container finished" podID="097ab5e1-b387-48d5-a9a5-c4a604bd98bc" containerID="db99aa6e4d7c1b427b0ae4000c38ccff5a9f3bbb304d1add4b5ca8d36b95263f" exitCode=255 Jul 23 14:56:49.164495 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.164470 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" event={"ID":"097ab5e1-b387-48d5-a9a5-c4a604bd98bc","Type":"ContainerDied","Data":"db99aa6e4d7c1b427b0ae4000c38ccff5a9f3bbb304d1add4b5ca8d36b95263f"} Jul 23 14:56:49.164548 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.164527 2582 scope.go:117] "RemoveContainer" containerID="8dc0504e9872400e98a856fd99a76e2968c5c38d561340ffd8a0af7c68072c85" Jul 23 14:56:49.164747 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.164727 2582 scope.go:117] "RemoveContainer" containerID="db99aa6e4d7c1b427b0ae4000c38ccff5a9f3bbb304d1add4b5ca8d36b95263f" Jul 23 14:56:49.164932 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:49.164912 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ctddv_openshift-console-operator(097ab5e1-b387-48d5-a9a5-c4a604bd98bc)\"" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" podUID="097ab5e1-b387-48d5-a9a5-c4a604bd98bc" Jul 23 14:56:49.185682 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.185645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqcpv\" (UniqueName: \"kubernetes.io/projected/5f04272a-404d-4370-96bb-019a750f3b49-kube-api-access-pqcpv\") pod \"migrator-7cc9c7ffd7-zk47q\" (UID: \"5f04272a-404d-4370-96bb-019a750f3b49\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" Jul 23 14:56:49.199156 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.199134 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqcpv\" (UniqueName: \"kubernetes.io/projected/5f04272a-404d-4370-96bb-019a750f3b49-kube-api-access-pqcpv\") pod \"migrator-7cc9c7ffd7-zk47q\" (UID: \"5f04272a-404d-4370-96bb-019a750f3b49\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" Jul 23 14:56:49.359844 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.359764 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" Jul 23 14:56:49.483151 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.483121 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q"] Jul 23 14:56:49.485731 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:56:49.485702 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f04272a_404d_4370_96bb_019a750f3b49.slice/crio-c026992df4d2c2146f48633b9200edbd9804106505f60a7a7def3f99dc80f113 WatchSource:0}: Error finding container c026992df4d2c2146f48633b9200edbd9804106505f60a7a7def3f99dc80f113: Status 404 returned error can't find the container with id c026992df4d2c2146f48633b9200edbd9804106505f60a7a7def3f99dc80f113 Jul 23 14:56:49.790951 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.790912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:49.791166 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:49.790969 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:49.791166 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:49.791047 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 14:56:49.791166 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:49.791078 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 14:56:49.791166 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:49.791113 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:51.791095809 +0000 UTC m=+259.942545156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : configmap "kube-rbac-proxy" not found Jul 23 14:56:49.791166 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:49.791135 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:51.791128571 +0000 UTC m=+259.942577918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : secret "insights-runtime-extractor-tls" not found Jul 23 14:56:50.089045 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.088965 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mfddd_d34f5e56-e8da-4a6a-9356-84a4ef68174a/dns-node-resolver/0.log" Jul 23 14:56:50.168867 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.168830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" event={"ID":"5f04272a-404d-4370-96bb-019a750f3b49","Type":"ContainerStarted","Data":"c026992df4d2c2146f48633b9200edbd9804106505f60a7a7def3f99dc80f113"} Jul 23 14:56:50.170323 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.170302 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 14:56:50.170611 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.170599 2582 scope.go:117] "RemoveContainer" containerID="db99aa6e4d7c1b427b0ae4000c38ccff5a9f3bbb304d1add4b5ca8d36b95263f" Jul 23 14:56:50.170758 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:50.170741 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ctddv_openshift-console-operator(097ab5e1-b387-48d5-a9a5-c4a604bd98bc)\"" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" podUID="097ab5e1-b387-48d5-a9a5-c4a604bd98bc" Jul 23 14:56:50.631942 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.631909 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-zgltj"] Jul 23 14:56:50.634666 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.634645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.637689 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.637671 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 23 14:56:50.637772 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.637725 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 23 14:56:50.639100 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.639082 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 23 14:56:50.639197 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.639098 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gtlhq\"" Jul 23 14:56:50.639197 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.639129 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 23 14:56:50.643250 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.643229 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-zgltj"] Jul 23 14:56:50.700944 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.700924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fd37617e-9064-4480-b744-cc1d9442dd4b-signing-cabundle\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.701036 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.701022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr9nq\" (UniqueName: \"kubernetes.io/projected/fd37617e-9064-4480-b744-cc1d9442dd4b-kube-api-access-cr9nq\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.701078 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.701056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fd37617e-9064-4480-b744-cc1d9442dd4b-signing-key\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.802045 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.801999 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fd37617e-9064-4480-b744-cc1d9442dd4b-signing-cabundle\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.802226 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.802109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr9nq\" (UniqueName: \"kubernetes.io/projected/fd37617e-9064-4480-b744-cc1d9442dd4b-kube-api-access-cr9nq\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.802226 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.802149 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fd37617e-9064-4480-b744-cc1d9442dd4b-signing-key\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.802849 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.802820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fd37617e-9064-4480-b744-cc1d9442dd4b-signing-cabundle\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.804651 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.804626 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fd37617e-9064-4480-b744-cc1d9442dd4b-signing-key\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.811021 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.810998 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr9nq\" (UniqueName: \"kubernetes.io/projected/fd37617e-9064-4480-b744-cc1d9442dd4b-kube-api-access-cr9nq\") pod \"service-ca-58598695f7-zgltj\" (UID: \"fd37617e-9064-4480-b744-cc1d9442dd4b\") " pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:50.889349 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.889273 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8pjfm_bb31c270-e90d-4a01-99e1-5c3d39e63631/node-ca/0.log" Jul 23 14:56:50.955938 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:50.955906 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-zgltj" Jul 23 14:56:51.076973 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.076945 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-zgltj"] Jul 23 14:56:51.079419 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:56:51.079396 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd37617e_9064_4480_b744_cc1d9442dd4b.slice/crio-6269f6f18cead41d26cca7076744037e60ad2a07e8a8751a05e76c33a4779cec WatchSource:0}: Error finding container 6269f6f18cead41d26cca7076744037e60ad2a07e8a8751a05e76c33a4779cec: Status 404 returned error can't find the container with id 6269f6f18cead41d26cca7076744037e60ad2a07e8a8751a05e76c33a4779cec Jul 23 14:56:51.176320 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.176288 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" event={"ID":"5f04272a-404d-4370-96bb-019a750f3b49","Type":"ContainerStarted","Data":"2530da63a2b0bc6a9adb058bbc55d1740f71bc00be9cf97f700a001262b51d87"} Jul 23 14:56:51.176805 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.176348 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" event={"ID":"5f04272a-404d-4370-96bb-019a750f3b49","Type":"ContainerStarted","Data":"1903a7a0e1dbd108f65fa6332d1302c1f802b820f7b1a7e5f02fe27a48acd579"} Jul 23 14:56:51.177755 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.177732 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-zgltj" event={"ID":"fd37617e-9064-4480-b744-cc1d9442dd4b","Type":"ContainerStarted","Data":"31d3bc2ed74a8d601d9af64e21e192e4f2e276c001d64544a581f56a629347c3"} Jul 23 14:56:51.177849 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.177770 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-zgltj" event={"ID":"fd37617e-9064-4480-b744-cc1d9442dd4b","Type":"ContainerStarted","Data":"6269f6f18cead41d26cca7076744037e60ad2a07e8a8751a05e76c33a4779cec"} Jul 23 14:56:51.192897 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.192857 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zk47q" podStartSLOduration=1.0655308589999999 podStartE2EDuration="2.192843612s" totalImagesPullingTime="1.127312753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:56:51.192243135 +0000 UTC m=+259.343692504" watchObservedRunningTime="2026-07-23 14:56:51.192843612 +0000 UTC m=+259.344293006" Jul 23 14:56:51.208027 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.207989 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-zgltj" podStartSLOduration=1.207976943 podStartE2EDuration="1.207976943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:56:51.207170214 +0000 UTC m=+259.358619583" watchObservedRunningTime="2026-07-23 14:56:51.207976943 +0000 UTC m=+259.359426311" Jul 23 14:56:51.689202 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.689169 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-67wsc_c15231de-2b65-4e56-8295-96311beed997/serve-healthcheck-canary/0.log" Jul 23 14:56:51.810105 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.810062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:51.810331 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.810124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:51.810331 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:51.810239 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 14:56:51.810331 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:51.810300 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 14:56:51.810331 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:51.810326 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:55.810311958 +0000 UTC m=+263.961761305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : secret "insights-runtime-extractor-tls" not found Jul 23 14:56:51.810521 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:51.810366 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:56:55.810348961 +0000 UTC m=+263.961798329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : configmap "kube-rbac-proxy" not found Jul 23 14:56:51.911332 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:51.911295 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:51.911493 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:51.911475 2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:51.911569 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:51.911557 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls podName:f90fd981-0d3a-4d04-adda-23df0f99ea87 nodeName:}" failed. No retries permitted until 2026-07-23 14:56:59.911535249 +0000 UTC m=+268.062984605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-58sm5" (UID: "f90fd981-0d3a-4d04-adda-23df0f99ea87") : secret "cluster-monitoring-operator-tls" not found Jul 23 14:56:52.012073 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:52.011986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:52.012227 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:52.012078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:56:52.012227 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:52.012108 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 14:56:52.012227 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:52.012171 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:57:00.012156154 +0000 UTC m=+268.163605502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : secret "router-metrics-certs-default" not found Jul 23 14:56:52.012227 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:52.012219 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle podName:46c81778-15ff-474c-a0f3-10a6458b470a nodeName:}" failed. No retries permitted until 2026-07-23 14:57:00.012207993 +0000 UTC m=+268.163657341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle") pod "router-default-85f8bf867b-ckpfm" (UID: "46c81778-15ff-474c-a0f3-10a6458b470a") : configmap references non-existent config key: service-ca.crt Jul 23 14:56:54.549011 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:54.548922 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:54.549011 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:54.548983 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:56:54.549432 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:54.549326 2582 scope.go:117] "RemoveContainer" containerID="db99aa6e4d7c1b427b0ae4000c38ccff5a9f3bbb304d1add4b5ca8d36b95263f" Jul 23 14:56:54.549479 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:54.549459 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ctddv_openshift-console-operator(097ab5e1-b387-48d5-a9a5-c4a604bd98bc)\"" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" podUID="097ab5e1-b387-48d5-a9a5-c4a604bd98bc" Jul 23 14:56:55.843868 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:55.843835 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:55.844295 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:55.843882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:56:55.844295 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:55.843962 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 14:56:55.844295 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:55.843995 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 14:56:55.844295 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:55.844020 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:57:03.844004252 +0000 UTC m=+271.995453624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : configmap "kube-rbac-proxy" not found Jul 23 14:56:55.844295 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:56:55.844052 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls podName:b959cf26-685b-48c2-9b92-8969331ab09a nodeName:}" failed. No retries permitted until 2026-07-23 14:57:03.844037825 +0000 UTC m=+271.995487172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96lf4" (UID: "b959cf26-685b-48c2-9b92-8969331ab09a") : secret "insights-runtime-extractor-tls" not found Jul 23 14:56:59.976239 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:59.976183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:56:59.978740 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:56:59.978718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f90fd981-0d3a-4d04-adda-23df0f99ea87-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-58sm5\" (UID: \"f90fd981-0d3a-4d04-adda-23df0f99ea87\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:57:00.077148 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.077103 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:00.077333 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.077215 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:00.077815 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.077790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46c81778-15ff-474c-a0f3-10a6458b470a-service-ca-bundle\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:00.079588 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.079568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46c81778-15ff-474c-a0f3-10a6458b470a-metrics-certs\") pod \"router-default-85f8bf867b-ckpfm\" (UID: \"46c81778-15ff-474c-a0f3-10a6458b470a\") " pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:00.152987 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.152949 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" Jul 23 14:57:00.233375 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.233280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:00.303575 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.301902 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5"] Jul 23 14:57:00.382128 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:00.382093 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85f8bf867b-ckpfm"] Jul 23 14:57:00.384906 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:00.384880 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c81778_15ff_474c_a0f3_10a6458b470a.slice/crio-3c56fc451d8e8bad283a5dd6fafd43fc45775f91d582a39ed1e38b168371179f WatchSource:0}: Error finding container 3c56fc451d8e8bad283a5dd6fafd43fc45775f91d582a39ed1e38b168371179f: Status 404 returned error can't find the container with id 3c56fc451d8e8bad283a5dd6fafd43fc45775f91d582a39ed1e38b168371179f Jul 23 14:57:01.212216 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:01.212179 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" event={"ID":"46c81778-15ff-474c-a0f3-10a6458b470a","Type":"ContainerStarted","Data":"45e7d8b08ef9306df9f2a4c62b910c353f682a3400e5596a6841fcde293afc21"} Jul 23 14:57:01.212704 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:01.212229 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" event={"ID":"46c81778-15ff-474c-a0f3-10a6458b470a","Type":"ContainerStarted","Data":"3c56fc451d8e8bad283a5dd6fafd43fc45775f91d582a39ed1e38b168371179f"} Jul 23 14:57:01.213385 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:01.213362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" event={"ID":"f90fd981-0d3a-4d04-adda-23df0f99ea87","Type":"ContainerStarted","Data":"0e6a162e8f7c26c6eda585305a1aae2085878d5d89004b536b4931982a602aa5"} Jul 23 14:57:01.232711 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:01.232666 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" podStartSLOduration=17.232652684 podStartE2EDuration="17.232652684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:01.231910693 +0000 UTC m=+269.383360062" watchObservedRunningTime="2026-07-23 14:57:01.232652684 +0000 UTC m=+269.384102052" Jul 23 14:57:01.233491 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:01.233470 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:01.235930 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:01.235910 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:02.217269 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:02.217222 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:02.218619 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:02.218595 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-85f8bf867b-ckpfm" Jul 23 14:57:03.222141 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:03.222099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" event={"ID":"f90fd981-0d3a-4d04-adda-23df0f99ea87","Type":"ContainerStarted","Data":"0175f8b2979b90160bff81439707889c32985e8063b95a557a995b85d12210f1"} Jul 23 14:57:03.239988 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:03.239933 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-58sm5" podStartSLOduration=16.887872636 podStartE2EDuration="19.239917556s" totalImagesPullingTime="2.35204492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:03.238719173 +0000 UTC m=+271.390168544" watchObservedRunningTime="2026-07-23 14:57:03.239917556 +0000 UTC m=+271.391366924" Jul 23 14:57:03.913741 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:03.913699 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:57:03.913741 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:03.913749 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:57:03.914242 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:03.914210 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b959cf26-685b-48c2-9b92-8969331ab09a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:57:03.916231 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:03.916205 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b959cf26-685b-48c2-9b92-8969331ab09a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96lf4\" (UID: \"b959cf26-685b-48c2-9b92-8969331ab09a\") " pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:57:04.002381 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:04.002338 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-96lf4" Jul 23 14:57:04.137559 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:04.137473 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-96lf4"] Jul 23 14:57:04.140115 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:04.140090 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb959cf26_685b_48c2_9b92_8969331ab09a.slice/crio-3f0be5287474b69ef02b3fc5fb3ab0a2107584611ab8bc29b2bd6e226e0ecdf8 WatchSource:0}: Error finding container 3f0be5287474b69ef02b3fc5fb3ab0a2107584611ab8bc29b2bd6e226e0ecdf8: Status 404 returned error can't find the container with id 3f0be5287474b69ef02b3fc5fb3ab0a2107584611ab8bc29b2bd6e226e0ecdf8 Jul 23 14:57:04.225760 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:04.225668 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96lf4" event={"ID":"b959cf26-685b-48c2-9b92-8969331ab09a","Type":"ContainerStarted","Data":"3f0be5287474b69ef02b3fc5fb3ab0a2107584611ab8bc29b2bd6e226e0ecdf8"} Jul 23 14:57:06.288643 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:06.288613 2582 scope.go:117] "RemoveContainer" containerID="db99aa6e4d7c1b427b0ae4000c38ccff5a9f3bbb304d1add4b5ca8d36b95263f" Jul 23 14:57:09.244799 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:09.244764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96lf4" event={"ID":"b959cf26-685b-48c2-9b92-8969331ab09a","Type":"ContainerStarted","Data":"c716c742c246a4602ce51faf70a1c5d598e7c1fc492b7e3b41375acec8f706bd"} Jul 23 14:57:09.246655 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:09.246633 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 14:57:09.246772 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:09.246682 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" event={"ID":"097ab5e1-b387-48d5-a9a5-c4a604bd98bc","Type":"ContainerStarted","Data":"2c58182055dab5f47cc35a8ea30e51f073571734fcd5b74382b794cbef41b89c"} Jul 23 14:57:09.247001 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:09.246979 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:57:09.264730 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:09.264667 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" podStartSLOduration=22.639254973 podStartE2EDuration="25.26464774s" totalImagesPullingTime="2.625392767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:09.264055225 +0000 UTC m=+277.415504595" watchObservedRunningTime="2026-07-23 14:57:09.26464774 +0000 UTC m=+277.416097114" Jul 23 14:57:09.276673 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:09.276648 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-ctddv" Jul 23 14:57:13.262459 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:13.262418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96lf4" event={"ID":"b959cf26-685b-48c2-9b92-8969331ab09a","Type":"ContainerStarted","Data":"15b2f4dc61003921ceff168303f8438680f785dbbd0446c03ccac7f52949ec9d"} Jul 23 14:57:14.914799 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:14.914760 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" podUID="08c05031-e76c-49da-94f0-a6329db3d717" Jul 23 14:57:14.914799 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:14.914760 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" Jul 23 14:57:14.915225 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:14.914761 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ld8p4" podUID="9b3ffade-f266-426b-be05-2e4ee4eb13db" Jul 23 14:57:15.270903 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:15.270826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:57:15.271073 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:15.270822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld8p4" Jul 23 14:57:15.271073 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:15.270835 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:57:16.275757 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:16.275665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96lf4" event={"ID":"b959cf26-685b-48c2-9b92-8969331ab09a","Type":"ContainerStarted","Data":"b4884d409da58aba3a49a50882596a2e7239ab3581b5ec4278f76f3e5559391a"} Jul 23 14:57:16.298685 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:16.295080 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-96lf4" podStartSLOduration=16.649855703 podStartE2EDuration="28.295055765s" totalImagesPullingTime="11.645200062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:56:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:16.292642522 +0000 UTC m=+284.444091893" watchObservedRunningTime="2026-07-23 14:57:16.295055765 +0000 UTC m=+284.446505136" Jul 23 14:57:18.191525 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.191491 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9"] Jul 23 14:57:18.194793 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.194771 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.198872 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.198845 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 14:57:18.198872 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.198852 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-grd95\"" Jul 23 14:57:18.199046 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.198852 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 14:57:18.199046 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.198895 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 14:57:18.203738 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.203713 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9"] Jul 23 14:57:18.246603 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.246574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:57:18.246795 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.246686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:57:18.246795 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.246717 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:57:18.248968 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.248944 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b3ffade-f266-426b-be05-2e4ee4eb13db-metrics-tls\") pod \"dns-default-ld8p4\" (UID: \"9b3ffade-f266-426b-be05-2e4ee4eb13db\") " pod="openshift-dns/dns-default-ld8p4" Jul 23 14:57:18.249128 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.249110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"image-registry-7dc6cc5c64-6rhfp\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:57:18.249298 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.249249 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/08c05031-e76c-49da-94f0-a6329db3d717-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9lsg8\" (UID: \"08c05031-e76c-49da-94f0-a6329db3d717\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:57:18.275075 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.275041 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zzbs4\"" Jul 23 14:57:18.275220 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.275113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b8t5d\"" Jul 23 14:57:18.275220 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.275119 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9zwgm\"" Jul 23 14:57:18.282380 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.282359 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" Jul 23 14:57:18.282497 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.282417 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:57:18.282497 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.282438 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld8p4" Jul 23 14:57:18.348022 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.347985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/819172c8-4d69-4e96-a60a-2dffbecf486a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.348141 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.348063 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.348141 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.348101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n94n\" (UniqueName: \"kubernetes.io/projected/819172c8-4d69-4e96-a60a-2dffbecf486a-kube-api-access-8n94n\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.348208 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.348160 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.446147 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.446119 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dc6cc5c64-6rhfp"] Jul 23 14:57:18.448868 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.448839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/819172c8-4d69-4e96-a60a-2dffbecf486a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.448997 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.448899 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.448997 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.448930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n94n\" (UniqueName: \"kubernetes.io/projected/819172c8-4d69-4e96-a60a-2dffbecf486a-kube-api-access-8n94n\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.448997 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.448974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.449151 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:18.449063 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4c2533_1907_4fdc_aa02_4dcb7d526917.slice/crio-31ec0f9d56b6ed58568982d2dfd192d3d46428567dc5871937ff6be73f5594f1 WatchSource:0}: Error finding container 31ec0f9d56b6ed58568982d2dfd192d3d46428567dc5871937ff6be73f5594f1: Status 404 returned error can't find the container with id 31ec0f9d56b6ed58568982d2dfd192d3d46428567dc5871937ff6be73f5594f1 Jul 23 14:57:18.449151 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:18.449087 2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 23 14:57:18.449151 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:18.449149 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls podName:819172c8-4d69-4e96-a60a-2dffbecf486a nodeName:}" failed. No retries permitted until 2026-07-23 14:57:18.94913076 +0000 UTC m=+287.100580122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-vpdv9" (UID: "819172c8-4d69-4e96-a60a-2dffbecf486a") : secret "prometheus-operator-tls" not found Jul 23 14:57:18.449655 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.449632 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/819172c8-4d69-4e96-a60a-2dffbecf486a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.451653 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.451630 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.457435 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.457407 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n94n\" (UniqueName: \"kubernetes.io/projected/819172c8-4d69-4e96-a60a-2dffbecf486a-kube-api-access-8n94n\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.667320 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.667292 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8"] Jul 23 14:57:18.669843 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.669815 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ld8p4"] Jul 23 14:57:18.670015 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:18.669990 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c05031_e76c_49da_94f0_a6329db3d717.slice/crio-eff3f00ac6a53deedde34dcfa5fa7801d3ef2dea1a9155e3b529ef28c8a758a8 WatchSource:0}: Error finding container eff3f00ac6a53deedde34dcfa5fa7801d3ef2dea1a9155e3b529ef28c8a758a8: Status 404 returned error can't find the container with id eff3f00ac6a53deedde34dcfa5fa7801d3ef2dea1a9155e3b529ef28c8a758a8 Jul 23 14:57:18.673153 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:18.673129 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3ffade_f266_426b_be05_2e4ee4eb13db.slice/crio-61442fb9a74bb377a23ce859d1f8da084949cda7c6200c9489ba06f0cc4e8ed1 WatchSource:0}: Error finding container 61442fb9a74bb377a23ce859d1f8da084949cda7c6200c9489ba06f0cc4e8ed1: Status 404 returned error can't find the container with id 61442fb9a74bb377a23ce859d1f8da084949cda7c6200c9489ba06f0cc4e8ed1 Jul 23 14:57:18.953219 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:18.953186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:18.953404 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:18.953348 2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 23 14:57:18.953445 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:18.953411 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls podName:819172c8-4d69-4e96-a60a-2dffbecf486a nodeName:}" failed. No retries permitted until 2026-07-23 14:57:19.953394112 +0000 UTC m=+288.104843458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-vpdv9" (UID: "819172c8-4d69-4e96-a60a-2dffbecf486a") : secret "prometheus-operator-tls" not found Jul 23 14:57:19.289111 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.289031 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" event={"ID":"1a4c2533-1907-4fdc-aa02-4dcb7d526917","Type":"ContainerStarted","Data":"265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168"} Jul 23 14:57:19.289111 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.289092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" event={"ID":"1a4c2533-1907-4fdc-aa02-4dcb7d526917","Type":"ContainerStarted","Data":"31ec0f9d56b6ed58568982d2dfd192d3d46428567dc5871937ff6be73f5594f1"} Jul 23 14:57:19.289586 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.289265 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:57:19.291187 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.291161 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld8p4" event={"ID":"9b3ffade-f266-426b-be05-2e4ee4eb13db","Type":"ContainerStarted","Data":"61442fb9a74bb377a23ce859d1f8da084949cda7c6200c9489ba06f0cc4e8ed1"} Jul 23 14:57:19.292582 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.292556 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" event={"ID":"08c05031-e76c-49da-94f0-a6329db3d717","Type":"ContainerStarted","Data":"eff3f00ac6a53deedde34dcfa5fa7801d3ef2dea1a9155e3b529ef28c8a758a8"} Jul 23 14:57:19.311388 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.310666 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" podStartSLOduration=287.310646339 podStartE2EDuration="4m47.310646339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:19.308838587 +0000 UTC m=+287.460287957" watchObservedRunningTime="2026-07-23 14:57:19.310646339 +0000 UTC m=+287.462095710" Jul 23 14:57:19.962423 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.962400 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:19.964876 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:19.964853 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/819172c8-4d69-4e96-a60a-2dffbecf486a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vpdv9\" (UID: \"819172c8-4d69-4e96-a60a-2dffbecf486a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:20.006396 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:20.006367 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" Jul 23 14:57:20.138296 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:20.138249 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9"] Jul 23 14:57:20.140519 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:20.140487 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819172c8_4d69_4e96_a60a_2dffbecf486a.slice/crio-4df7bdf6458368733606fbfd1e27d19919f9ed49e98a716da8d0015439c80e17 WatchSource:0}: Error finding container 4df7bdf6458368733606fbfd1e27d19919f9ed49e98a716da8d0015439c80e17: Status 404 returned error can't find the container with id 4df7bdf6458368733606fbfd1e27d19919f9ed49e98a716da8d0015439c80e17 Jul 23 14:57:20.297580 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:20.297499 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" event={"ID":"08c05031-e76c-49da-94f0-a6329db3d717","Type":"ContainerStarted","Data":"7aa0d97ca907954fdb97282b944a1b745476c1d542c9f04c0b20493f89971b12"} Jul 23 14:57:20.298967 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:20.298942 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" event={"ID":"819172c8-4d69-4e96-a60a-2dffbecf486a","Type":"ContainerStarted","Data":"4df7bdf6458368733606fbfd1e27d19919f9ed49e98a716da8d0015439c80e17"} Jul 23 14:57:20.314598 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:20.314540 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9lsg8" podStartSLOduration=275.06625751 podStartE2EDuration="4m36.31452052s" totalImagesPullingTime="1.24826301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:20.313391901 +0000 UTC m=+288.464841271" watchObservedRunningTime="2026-07-23 14:57:20.31452052 +0000 UTC m=+288.465969892" Jul 23 14:57:21.304951 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:21.304911 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld8p4" event={"ID":"9b3ffade-f266-426b-be05-2e4ee4eb13db","Type":"ContainerStarted","Data":"b180ef77db31a46bac6063fe4296181374d0383e18da80aad458438b6f8b96e3"} Jul 23 14:57:22.309214 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:22.309094 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" event={"ID":"819172c8-4d69-4e96-a60a-2dffbecf486a","Type":"ContainerStarted","Data":"c419cda5094102e6a3b47eb6a70737847d54fb003e67f9c662fcb0bf138447ef"} Jul 23 14:57:22.310852 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:22.310828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld8p4" event={"ID":"9b3ffade-f266-426b-be05-2e4ee4eb13db","Type":"ContainerStarted","Data":"01575543ba8471ccea57ff25c07fc6d92b45ac61b725605be253cc83584c0a5a"} Jul 23 14:57:22.310958 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:22.310952 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ld8p4" Jul 23 14:57:22.329314 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:22.329249 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ld8p4" podStartSLOduration=251.851305964 podStartE2EDuration="4m14.329234806s" totalImagesPullingTime="2.477928842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:22.327533134 +0000 UTC m=+290.478982503" watchObservedRunningTime="2026-07-23 14:57:22.329234806 +0000 UTC m=+290.480684174" Jul 23 14:57:23.315896 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:23.315854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" event={"ID":"819172c8-4d69-4e96-a60a-2dffbecf486a","Type":"ContainerStarted","Data":"ed378d98878aa22e80c0fe5a44c5aaa1e53de60ba702ed198abae2ab236839c6"} Jul 23 14:57:23.332313 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:23.332249 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vpdv9" podStartSLOduration=3.284303688 podStartE2EDuration="5.332234143s" totalImagesPullingTime="2.047930455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:23.331459551 +0000 UTC m=+291.482908920" watchObservedRunningTime="2026-07-23 14:57:23.332234143 +0000 UTC m=+291.483683510" Jul 23 14:57:25.554618 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.554582 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kkpr5"] Jul 23 14:57:25.559447 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.559423 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.563862 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.563818 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 14:57:25.564014 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.563994 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 14:57:25.564219 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.564203 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 14:57:25.564421 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.564404 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-62kkx\"" Jul 23 14:57:25.608489 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.608459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-root\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.608677 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.608498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-textfile\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.608784 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.608515 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-tls\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.609076 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.609039 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64b52a1-64f5-4159-9927-d77fd6f06584-metrics-client-ca\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.609288 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.609268 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7s5f\" (UniqueName: \"kubernetes.io/projected/a64b52a1-64f5-4159-9927-d77fd6f06584-kube-api-access-h7s5f\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.609594 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.609573 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-wtmp\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.609743 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.609726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.609872 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.609857 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-accelerators-collector-config\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.609993 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.609980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-sys\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711073 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711033 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7s5f\" (UniqueName: \"kubernetes.io/projected/a64b52a1-64f5-4159-9927-d77fd6f06584-kube-api-access-h7s5f\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711284 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-wtmp\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711284 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711407 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-accelerators-collector-config\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711407 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-sys\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711516 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711320 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-wtmp\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711516 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-root\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711609 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711596 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-textfile\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711661 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711481 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-sys\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711716 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711532 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a64b52a1-64f5-4159-9927-d77fd6f06584-root\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711893 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711862 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-accelerators-collector-config\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711947 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711924 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-textfile\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.711992 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.711981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-tls\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.712029 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.712020 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64b52a1-64f5-4159-9927-d77fd6f06584-metrics-client-ca\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.712611 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.712538 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a64b52a1-64f5-4159-9927-d77fd6f06584-metrics-client-ca\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.714068 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.714034 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.714469 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.714446 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a64b52a1-64f5-4159-9927-d77fd6f06584-node-exporter-tls\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.719817 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.719797 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7s5f\" (UniqueName: \"kubernetes.io/projected/a64b52a1-64f5-4159-9927-d77fd6f06584-kube-api-access-h7s5f\") pod \"node-exporter-kkpr5\" (UID: \"a64b52a1-64f5-4159-9927-d77fd6f06584\") " pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.869498 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:25.869422 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kkpr5" Jul 23 14:57:25.881225 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:25.881186 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64b52a1_64f5_4159_9927_d77fd6f06584.slice/crio-6918e34b3fcedc7a2d2362a2a788639275e45e804b2f26867518ac2aba0962bf WatchSource:0}: Error finding container 6918e34b3fcedc7a2d2362a2a788639275e45e804b2f26867518ac2aba0962bf: Status 404 returned error can't find the container with id 6918e34b3fcedc7a2d2362a2a788639275e45e804b2f26867518ac2aba0962bf Jul 23 14:57:26.326329 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:26.326297 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kkpr5" event={"ID":"a64b52a1-64f5-4159-9927-d77fd6f06584","Type":"ContainerStarted","Data":"6918e34b3fcedc7a2d2362a2a788639275e45e804b2f26867518ac2aba0962bf"} Jul 23 14:57:27.331498 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:27.331414 2582 generic.go:356] "Generic (PLEG): container finished" podID="a64b52a1-64f5-4159-9927-d77fd6f06584" containerID="d4b46bb6d578cb5ecf59b11d5aede3e698eeef0e3d2bfb3323e5d9955406c894" exitCode=0 Jul 23 14:57:27.331498 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:27.331469 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kkpr5" event={"ID":"a64b52a1-64f5-4159-9927-d77fd6f06584","Type":"ContainerDied","Data":"d4b46bb6d578cb5ecf59b11d5aede3e698eeef0e3d2bfb3323e5d9955406c894"} Jul 23 14:57:28.337325 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:28.337294 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kkpr5" event={"ID":"a64b52a1-64f5-4159-9927-d77fd6f06584","Type":"ContainerStarted","Data":"8336b293be6142e3db1d472d1a3fa408194efdffb1b49dff63973df869b6c86a"} Jul 23 14:57:28.337705 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:28.337344 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kkpr5" event={"ID":"a64b52a1-64f5-4159-9927-d77fd6f06584","Type":"ContainerStarted","Data":"e4bfcfdc200cd693ff0792d921b903ccfff08d191490181c0d1584bdf3c4b575"} Jul 23 14:57:28.359202 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:28.359153 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kkpr5" podStartSLOduration=2.247212382 podStartE2EDuration="3.359140057s" totalImagesPullingTime="1.041205525s" totalInitContainerRuntime="70.72215ms" isStatefulPod=true podCreationTimestamp="2026-07-23 14:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:28.356959018 +0000 UTC m=+296.508408388" watchObservedRunningTime="2026-07-23 14:57:28.359140057 +0000 UTC m=+296.510589426" Jul 23 14:57:29.943486 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.943446 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7bcf78d555-7rvnd"] Jul 23 14:57:29.946754 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.946732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:29.949646 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.949627 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 14:57:29.950843 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.950820 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 23 14:57:29.950843 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.950837 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 23 14:57:29.951005 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.950884 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f55sk4hjpelkb\"" Jul 23 14:57:29.951005 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.950896 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 23 14:57:29.951005 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.950923 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6sdfw\"" Jul 23 14:57:29.958135 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:29.958115 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bcf78d555-7rvnd"] Jul 23 14:57:30.050429 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zxbw\" (UniqueName: \"kubernetes.io/projected/7083e62d-5c57-4533-848c-92a945ee2230-kube-api-access-5zxbw\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.050591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7083e62d-5c57-4533-848c-92a945ee2230-metrics-server-audit-profiles\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.050591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050489 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-secret-metrics-server-tls\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.050591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7083e62d-5c57-4533-848c-92a945ee2230-audit-log\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.050591 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050528 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-client-ca-bundle\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.050717 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050606 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7083e62d-5c57-4533-848c-92a945ee2230-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.050717 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.050635 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-secret-metrics-server-client-certs\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151408 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-client-ca-bundle\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151581 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151421 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7083e62d-5c57-4533-848c-92a945ee2230-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151581 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151542 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-secret-metrics-server-client-certs\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151667 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zxbw\" (UniqueName: \"kubernetes.io/projected/7083e62d-5c57-4533-848c-92a945ee2230-kube-api-access-5zxbw\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151667 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7083e62d-5c57-4533-848c-92a945ee2230-metrics-server-audit-profiles\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-secret-metrics-server-tls\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.151763 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.151714 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7083e62d-5c57-4533-848c-92a945ee2230-audit-log\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.152073 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.152041 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7083e62d-5c57-4533-848c-92a945ee2230-audit-log\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.152239 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.152143 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7083e62d-5c57-4533-848c-92a945ee2230-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.152532 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.152510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7083e62d-5c57-4533-848c-92a945ee2230-metrics-server-audit-profiles\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.153992 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.153969 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-client-ca-bundle\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.154103 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.154045 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-secret-metrics-server-client-certs\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.154163 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.154110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7083e62d-5c57-4533-848c-92a945ee2230-secret-metrics-server-tls\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.160357 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.160329 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zxbw\" (UniqueName: \"kubernetes.io/projected/7083e62d-5c57-4533-848c-92a945ee2230-kube-api-access-5zxbw\") pod \"metrics-server-7bcf78d555-7rvnd\" (UID: \"7083e62d-5c57-4533-848c-92a945ee2230\") " pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.255426 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.255354 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:30.289181 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:57:30.289131 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xjwws" podUID="3d60ecea-1743-434c-8b1c-78b027daac9b" Jul 23 14:57:30.399541 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.399507 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bcf78d555-7rvnd"] Jul 23 14:57:30.403026 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:30.402997 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7083e62d_5c57_4533_848c_92a945ee2230.slice/crio-09810f78c52713cf5cdf5cf1e307a4505423a0d3aa47c2cf094243e336d016b9 WatchSource:0}: Error finding container 09810f78c52713cf5cdf5cf1e307a4505423a0d3aa47c2cf094243e336d016b9: Status 404 returned error can't find the container with id 09810f78c52713cf5cdf5cf1e307a4505423a0d3aa47c2cf094243e336d016b9 Jul 23 14:57:30.760870 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.760834 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf"] Jul 23 14:57:30.765556 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.765534 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.768421 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.768400 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 14:57:30.768553 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.768457 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 14:57:30.768553 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.768528 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rpnfk\"" Jul 23 14:57:30.768687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.768607 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 14:57:30.768687 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.768624 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 14:57:30.768791 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.768754 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 14:57:30.773171 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.773135 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 14:57:30.777111 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.777091 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf"] Jul 23 14:57:30.858486 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858451 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgqh8\" (UniqueName: \"kubernetes.io/projected/80cd24b9-428c-446d-88da-ee579369c459-kube-api-access-fgqh8\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858667 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858500 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858667 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858554 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-serving-certs-ca-bundle\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858667 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858596 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858667 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-metrics-client-ca\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858862 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858695 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-telemeter-client-tls\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858862 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-federate-client-tls\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.858862 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.858784 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-secret-telemeter-client\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.959671 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959636 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-metrics-client-ca\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.959671 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959670 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-telemeter-client-tls\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-federate-client-tls\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-secret-telemeter-client\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgqh8\" (UniqueName: \"kubernetes.io/projected/80cd24b9-428c-446d-88da-ee579369c459-kube-api-access-fgqh8\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-serving-certs-ca-bundle\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960175 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.959837 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.960559 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-metrics-client-ca\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.960914 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.960893 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.961214 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.961183 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cd24b9-428c-446d-88da-ee579369c459-serving-certs-ca-bundle\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.962640 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.962615 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-telemeter-client-tls\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.962717 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.962685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-secret-telemeter-client\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.962717 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.962696 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-federate-client-tls\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.962779 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.962728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80cd24b9-428c-446d-88da-ee579369c459-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:30.968203 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:30.968178 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgqh8\" (UniqueName: \"kubernetes.io/projected/80cd24b9-428c-446d-88da-ee579369c459-kube-api-access-fgqh8\") pod \"telemeter-client-69dd4f8bc4-rwtpf\" (UID: \"80cd24b9-428c-446d-88da-ee579369c459\") " pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:31.013010 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.012940 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85f87446b6-whtbv"] Jul 23 14:57:31.016399 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.016379 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.019291 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019136 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 14:57:31.019291 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019203 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 14:57:31.019612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019295 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 14:57:31.019612 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019521 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 14:57:31.019859 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019839 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 14:57:31.019981 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019888 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 14:57:31.019981 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.019907 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 14:57:31.020635 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.020610 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ld9fz\"" Jul 23 14:57:31.024059 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.024040 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 14:57:31.027772 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.027742 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f87446b6-whtbv"] Jul 23 14:57:31.060241 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-trusted-ca-bundle\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.060398 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060247 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-oauth-serving-cert\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.060398 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkbbk\" (UniqueName: \"kubernetes.io/projected/8950e9f9-7e54-4502-a46a-366135a2dfc5-kube-api-access-hkbbk\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.060398 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060392 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-serving-cert\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.060548 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060416 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-oauth-config\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.060548 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060437 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-config\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.060548 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.060467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-service-ca\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.075134 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.075107 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" Jul 23 14:57:31.160996 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.160963 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-serving-cert\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.161160 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-oauth-config\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.161160 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161027 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-config\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.161405 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-service-ca\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.161546 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-trusted-ca-bundle\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.161546 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-oauth-serving-cert\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.162334 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161848 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-config\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.162334 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.161902 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbbk\" (UniqueName: \"kubernetes.io/projected/8950e9f9-7e54-4502-a46a-366135a2dfc5-kube-api-access-hkbbk\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.162334 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.162075 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-oauth-serving-cert\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.162528 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.162341 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-service-ca\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.162528 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.162468 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-trusted-ca-bundle\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.164590 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.164570 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-serving-cert\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.164912 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.164874 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-oauth-config\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.173290 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.172833 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbbk\" (UniqueName: \"kubernetes.io/projected/8950e9f9-7e54-4502-a46a-366135a2dfc5-kube-api-access-hkbbk\") pod \"console-85f87446b6-whtbv\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.229193 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.229133 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf"] Jul 23 14:57:31.232413 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:31.232383 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80cd24b9_428c_446d_88da_ee579369c459.slice/crio-7c90757add7a582bdc73f477aac52d1d6804350432cb4e4a0726e99a7233a65b WatchSource:0}: Error finding container 7c90757add7a582bdc73f477aac52d1d6804350432cb4e4a0726e99a7233a65b: Status 404 returned error can't find the container with id 7c90757add7a582bdc73f477aac52d1d6804350432cb4e4a0726e99a7233a65b Jul 23 14:57:31.329061 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.328985 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:31.351170 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.351135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" event={"ID":"80cd24b9-428c-446d-88da-ee579369c459","Type":"ContainerStarted","Data":"7c90757add7a582bdc73f477aac52d1d6804350432cb4e4a0726e99a7233a65b"} Jul 23 14:57:31.352650 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.352624 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" event={"ID":"7083e62d-5c57-4533-848c-92a945ee2230","Type":"ContainerStarted","Data":"09810f78c52713cf5cdf5cf1e307a4505423a0d3aa47c2cf094243e336d016b9"} Jul 23 14:57:31.483438 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:31.483358 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f87446b6-whtbv"] Jul 23 14:57:32.221347 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.221314 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 14:57:32.225631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.225600 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 14:57:32.237511 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.237488 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 14:57:32.319065 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.318895 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ld8p4" Jul 23 14:57:32.362744 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.362706 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" event={"ID":"7083e62d-5c57-4533-848c-92a945ee2230","Type":"ContainerStarted","Data":"f8e67829f3ecebffbc4ad3ef3295d8b265ada30793d8906dc94980f654637322"} Jul 23 14:57:32.364399 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.364358 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f87446b6-whtbv" event={"ID":"8950e9f9-7e54-4502-a46a-366135a2dfc5","Type":"ContainerStarted","Data":"cf2b4c6c6d4dfad2e5421ee027ee98b1c02fe9f3c5e62b83bc6f047d77a638fa"} Jul 23 14:57:32.380189 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:32.380141 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" podStartSLOduration=1.845599701 podStartE2EDuration="3.380124211s" totalImagesPullingTime="1.53452451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:32.379419054 +0000 UTC m=+300.530868425" watchObservedRunningTime="2026-07-23 14:57:32.380124211 +0000 UTC m=+300.531573580" Jul 23 14:57:33.508943 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:33.508915 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 14:57:34.375272 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:34.375231 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" event={"ID":"80cd24b9-428c-446d-88da-ee579369c459","Type":"ContainerStarted","Data":"d733210ac4e03eeb84efc86fc2909bbba77bf164252138d090add375a0edda76"} Jul 23 14:57:35.369463 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:35.369426 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7dc6cc5c64-6rhfp"] Jul 23 14:57:35.376076 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:35.375926 2582 patch_prober.go:28] interesting pod/image-registry-7dc6cc5c64-6rhfp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 14:57:35.376076 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:35.376001 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 14:57:35.381658 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:35.381629 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f87446b6-whtbv" event={"ID":"8950e9f9-7e54-4502-a46a-366135a2dfc5","Type":"ContainerStarted","Data":"0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9"} Jul 23 14:57:35.412450 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:35.412394 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85f87446b6-whtbv" podStartSLOduration=2.421229876 podStartE2EDuration="5.412380228s" totalImagesPullingTime="2.991150352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:35.411777041 +0000 UTC m=+303.563226410" watchObservedRunningTime="2026-07-23 14:57:35.412380228 +0000 UTC m=+303.563829596" Jul 23 14:57:36.387364 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:36.387317 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" event={"ID":"80cd24b9-428c-446d-88da-ee579369c459","Type":"ContainerStarted","Data":"594ec858d105d9215fa27ebe2a01ccf5abf4978fe4a9664723437d263ab21f65"} Jul 23 14:57:36.387794 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:36.387383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" event={"ID":"80cd24b9-428c-446d-88da-ee579369c459","Type":"ContainerStarted","Data":"af44087d3124a136d0bdc75b614d6d816b5fde5e3da765ab92f4768068438b93"} Jul 23 14:57:36.410031 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:36.409984 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69dd4f8bc4-rwtpf" podStartSLOduration=2.293712264 podStartE2EDuration="6.409967158s" totalImagesPullingTime="4.116254894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:57:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:36.408004736 +0000 UTC m=+304.559454108" watchObservedRunningTime="2026-07-23 14:57:36.409967158 +0000 UTC m=+304.561416570" Jul 23 14:57:37.070082 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.070040 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5984cc87c-fd775"] Jul 23 14:57:37.073451 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.073426 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.082483 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.082460 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5984cc87c-fd775"] Jul 23 14:57:37.113866 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.113826 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-serving-cert\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.114049 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.113874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-trusted-ca-bundle\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.114049 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.113899 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v77sk\" (UniqueName: \"kubernetes.io/projected/c741be1b-d719-4648-93f0-29355a2c5e4b-kube-api-access-v77sk\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.114049 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.114036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-service-ca\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.114203 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.114068 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-oauth-serving-cert\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.114203 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.114097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-console-config\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.114203 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.114151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-oauth-config\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215538 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215499 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-oauth-config\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215709 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-serving-cert\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215768 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215755 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-trusted-ca-bundle\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215803 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215779 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v77sk\" (UniqueName: \"kubernetes.io/projected/c741be1b-d719-4648-93f0-29355a2c5e4b-kube-api-access-v77sk\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215849 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215835 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-service-ca\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215900 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-oauth-serving-cert\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.215900 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.215879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-console-config\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.216658 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.216630 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-trusted-ca-bundle\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.216764 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.216657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-service-ca\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.216764 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.216657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-console-config\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.216834 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.216761 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-oauth-serving-cert\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.218119 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.218101 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-oauth-config\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.218286 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.218248 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-serving-cert\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.224474 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.224454 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v77sk\" (UniqueName: \"kubernetes.io/projected/c741be1b-d719-4648-93f0-29355a2c5e4b-kube-api-access-v77sk\") pod \"console-5984cc87c-fd775\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.384608 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.384517 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:37.515328 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:37.515306 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5984cc87c-fd775"] Jul 23 14:57:37.517708 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:57:37.517678 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc741be1b_d719_4648_93f0_29355a2c5e4b.slice/crio-ad77fa67f9a1f78267aa9aaa9bc55472c13b9c247bb9fe03709cf41285016e1e WatchSource:0}: Error finding container ad77fa67f9a1f78267aa9aaa9bc55472c13b9c247bb9fe03709cf41285016e1e: Status 404 returned error can't find the container with id ad77fa67f9a1f78267aa9aaa9bc55472c13b9c247bb9fe03709cf41285016e1e Jul 23 14:57:38.395701 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:38.395660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5984cc87c-fd775" event={"ID":"c741be1b-d719-4648-93f0-29355a2c5e4b","Type":"ContainerStarted","Data":"9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1"} Jul 23 14:57:38.395918 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:38.395717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5984cc87c-fd775" event={"ID":"c741be1b-d719-4648-93f0-29355a2c5e4b","Type":"ContainerStarted","Data":"ad77fa67f9a1f78267aa9aaa9bc55472c13b9c247bb9fe03709cf41285016e1e"} Jul 23 14:57:38.414390 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:38.414334 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5984cc87c-fd775" podStartSLOduration=1.414315295 podStartE2EDuration="1.414315295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:57:38.413797966 +0000 UTC m=+306.565247346" watchObservedRunningTime="2026-07-23 14:57:38.414315295 +0000 UTC m=+306.565764739" Jul 23 14:57:41.329451 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:41.329414 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:41.329936 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:41.329487 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:41.334104 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:41.334081 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:41.410899 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:41.410874 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:57:45.288531 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:45.288498 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:57:45.373855 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:45.373825 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:57:47.385394 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:47.385348 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:47.386016 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:47.385457 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:47.390169 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:47.390144 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:47.431995 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:47.431963 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:57:47.482818 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:47.482784 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f87446b6-whtbv"] Jul 23 14:57:50.256352 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:50.256321 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:57:50.256780 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:57:50.256379 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:58:00.391971 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.391920 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" containerName="registry" containerID="cri-o://265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168" gracePeriod=30 Jul 23 14:58:00.639182 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.639157 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:58:00.726748 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726713 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfgs6\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-kube-api-access-gfgs6\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.726923 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726817 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-image-registry-private-configuration\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.726923 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726841 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.726923 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726861 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-certificates\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.726923 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726892 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-installation-pull-secrets\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.726923 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726917 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-bound-sa-token\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.727148 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726948 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-trusted-ca\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.727148 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.726979 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a4c2533-1907-4fdc-aa02-4dcb7d526917-ca-trust-extracted\") pod \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\" (UID: \"1a4c2533-1907-4fdc-aa02-4dcb7d526917\") " Jul 23 14:58:00.727790 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.727652 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:58:00.727948 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.727915 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:58:00.729516 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.729490 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:58:00.729516 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.729497 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-kube-api-access-gfgs6" (OuterVolumeSpecName: "kube-api-access-gfgs6") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "kube-api-access-gfgs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:58:00.729701 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.729593 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:58:00.729883 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.729848 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:58:00.729948 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.729908 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:58:00.736038 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.736015 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c2533-1907-4fdc-aa02-4dcb7d526917-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1a4c2533-1907-4fdc-aa02-4dcb7d526917" (UID: "1a4c2533-1907-4fdc-aa02-4dcb7d526917"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 14:58:00.827897 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827850 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-image-registry-private-configuration\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.827897 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827890 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-tls\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.827897 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827901 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-registry-certificates\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.827897 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827911 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a4c2533-1907-4fdc-aa02-4dcb7d526917-installation-pull-secrets\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.828152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827920 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-bound-sa-token\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.828152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827928 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a4c2533-1907-4fdc-aa02-4dcb7d526917-trusted-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.828152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827936 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a4c2533-1907-4fdc-aa02-4dcb7d526917-ca-trust-extracted\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:00.828152 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:00.827944 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfgs6\" (UniqueName: \"kubernetes.io/projected/1a4c2533-1907-4fdc-aa02-4dcb7d526917-kube-api-access-gfgs6\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:01.480302 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.480249 2582 generic.go:356] "Generic (PLEG): container finished" podID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" containerID="265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168" exitCode=0 Jul 23 14:58:01.480749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.480314 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" Jul 23 14:58:01.480749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.480314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" event={"ID":"1a4c2533-1907-4fdc-aa02-4dcb7d526917","Type":"ContainerDied","Data":"265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168"} Jul 23 14:58:01.480749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.480434 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7dc6cc5c64-6rhfp" event={"ID":"1a4c2533-1907-4fdc-aa02-4dcb7d526917","Type":"ContainerDied","Data":"31ec0f9d56b6ed58568982d2dfd192d3d46428567dc5871937ff6be73f5594f1"} Jul 23 14:58:01.480749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.480462 2582 scope.go:117] "RemoveContainer" containerID="265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168" Jul 23 14:58:01.489434 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.489420 2582 scope.go:117] "RemoveContainer" containerID="265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168" Jul 23 14:58:01.489704 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:58:01.489685 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168\": container with ID starting with 265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168 not found: ID does not exist" containerID="265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168" Jul 23 14:58:01.489745 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.489713 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168"} err="failed to get container status \"265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168\": rpc error: code = NotFound desc = could not find container \"265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168\": container with ID starting with 265911dd3d29077ff0d8703c998c1e6af0560cb9dba7ac76a9e72870080cb168 not found: ID does not exist" Jul 23 14:58:01.502713 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.502682 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7dc6cc5c64-6rhfp"] Jul 23 14:58:01.506411 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:01.506383 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7dc6cc5c64-6rhfp"] Jul 23 14:58:02.294034 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:02.294000 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" path="/var/lib/kubelet/pods/1a4c2533-1907-4fdc-aa02-4dcb7d526917/volumes" Jul 23 14:58:08.509617 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:08.509583 2582 generic.go:356] "Generic (PLEG): container finished" podID="5acaf8bb-5254-4bba-97af-d0870a555d89" containerID="6fd633d92c9817c867178cae9e4b7a1cfa88a692d2a98ef4b15785c109ab72dd" exitCode=0 Jul 23 14:58:08.510031 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:08.509666 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" event={"ID":"5acaf8bb-5254-4bba-97af-d0870a555d89","Type":"ContainerDied","Data":"6fd633d92c9817c867178cae9e4b7a1cfa88a692d2a98ef4b15785c109ab72dd"} Jul 23 14:58:08.510031 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:08.509964 2582 scope.go:117] "RemoveContainer" containerID="6fd633d92c9817c867178cae9e4b7a1cfa88a692d2a98ef4b15785c109ab72dd" Jul 23 14:58:09.516071 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:09.516032 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hmpxd" event={"ID":"5acaf8bb-5254-4bba-97af-d0870a555d89","Type":"ContainerStarted","Data":"b756604d53937df9f93290299ffd79324a3a5b1a135939818c13e214d06d6360"} Jul 23 14:58:10.261561 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:10.261522 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:58:10.265360 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:10.265337 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7bcf78d555-7rvnd" Jul 23 14:58:12.504365 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.504320 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85f87446b6-whtbv" podUID="8950e9f9-7e54-4502-a46a-366135a2dfc5" containerName="console" containerID="cri-o://0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9" gracePeriod=15 Jul 23 14:58:12.759975 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.759918 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:58:12.830378 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830342 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-config\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830400 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-oauth-serving-cert\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830441 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-trusted-ca-bundle\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830466 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-serving-cert\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830500 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkbbk\" (UniqueName: \"kubernetes.io/projected/8950e9f9-7e54-4502-a46a-366135a2dfc5-kube-api-access-hkbbk\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830539 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830530 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-service-ca\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830795 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830577 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-oauth-config\") pod \"8950e9f9-7e54-4502-a46a-366135a2dfc5\" (UID: \"8950e9f9-7e54-4502-a46a-366135a2dfc5\") " Jul 23 14:58:12.830914 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830874 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:58:12.830914 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830880 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:58:12.831041 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.830944 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-service-ca" (OuterVolumeSpecName: "service-ca") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:58:12.831088 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.831047 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-config" (OuterVolumeSpecName: "console-config") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:58:12.832971 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.832948 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8950e9f9-7e54-4502-a46a-366135a2dfc5-kube-api-access-hkbbk" (OuterVolumeSpecName: "kube-api-access-hkbbk") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "kube-api-access-hkbbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:58:12.833097 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.832973 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:58:12.833097 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.833049 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8950e9f9-7e54-4502-a46a-366135a2dfc5" (UID: "8950e9f9-7e54-4502-a46a-366135a2dfc5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:58:12.931599 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931558 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-oauth-serving-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:12.931599 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931590 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-trusted-ca-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:12.931599 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931603 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-serving-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:12.931825 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931616 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkbbk\" (UniqueName: \"kubernetes.io/projected/8950e9f9-7e54-4502-a46a-366135a2dfc5-kube-api-access-hkbbk\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:12.931825 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931631 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:12.931825 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931641 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-oauth-config\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:12.931825 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:12.931652 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8950e9f9-7e54-4502-a46a-366135a2dfc5-console-config\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:58:13.532165 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.532130 2582 generic.go:356] "Generic (PLEG): container finished" podID="8950e9f9-7e54-4502-a46a-366135a2dfc5" containerID="0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9" exitCode=0 Jul 23 14:58:13.532625 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.532180 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f87446b6-whtbv" Jul 23 14:58:13.532625 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.532186 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f87446b6-whtbv" event={"ID":"8950e9f9-7e54-4502-a46a-366135a2dfc5","Type":"ContainerDied","Data":"0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9"} Jul 23 14:58:13.532625 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.532236 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f87446b6-whtbv" event={"ID":"8950e9f9-7e54-4502-a46a-366135a2dfc5","Type":"ContainerDied","Data":"cf2b4c6c6d4dfad2e5421ee027ee98b1c02fe9f3c5e62b83bc6f047d77a638fa"} Jul 23 14:58:13.532625 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.532288 2582 scope.go:117] "RemoveContainer" containerID="0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9" Jul 23 14:58:13.542706 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.541820 2582 scope.go:117] "RemoveContainer" containerID="0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9" Jul 23 14:58:13.543108 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:58:13.543084 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9\": container with ID starting with 0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9 not found: ID does not exist" containerID="0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9" Jul 23 14:58:13.543219 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.543154 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9"} err="failed to get container status \"0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9\": rpc error: code = NotFound desc = could not find container \"0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9\": container with ID starting with 0a8d116874e5f025ba51a0829fa7365797ca373e76a7d167c7a313d593b9eac9 not found: ID does not exist" Jul 23 14:58:13.561373 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.561345 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f87446b6-whtbv"] Jul 23 14:58:13.564835 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:13.564810 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85f87446b6-whtbv"] Jul 23 14:58:14.293207 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:14.293162 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8950e9f9-7e54-4502-a46a-366135a2dfc5" path="/var/lib/kubelet/pods/8950e9f9-7e54-4502-a46a-366135a2dfc5/volumes" Jul 23 14:58:46.318941 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.318905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:58:46.321386 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.321364 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d60ecea-1743-434c-8b1c-78b027daac9b-metrics-certs\") pod \"network-metrics-daemon-xjwws\" (UID: \"3d60ecea-1743-434c-8b1c-78b027daac9b\") " pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:58:46.492638 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.492599 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-brw86\"" Jul 23 14:58:46.499862 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.499842 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjwws" Jul 23 14:58:46.638374 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.638340 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xjwws"] Jul 23 14:58:46.640528 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:58:46.640492 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d60ecea_1743_434c_8b1c_78b027daac9b.slice/crio-ec3fba46133e33793377d3e1cab41a4d1878eb6ab06e3455f7f82dbb97034e2b WatchSource:0}: Error finding container ec3fba46133e33793377d3e1cab41a4d1878eb6ab06e3455f7f82dbb97034e2b: Status 404 returned error can't find the container with id ec3fba46133e33793377d3e1cab41a4d1878eb6ab06e3455f7f82dbb97034e2b Jul 23 14:58:46.643408 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643383 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55f9596797-w5hdr"] Jul 23 14:58:46.643804 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643786 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8950e9f9-7e54-4502-a46a-366135a2dfc5" containerName="console" Jul 23 14:58:46.643870 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643807 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8950e9f9-7e54-4502-a46a-366135a2dfc5" containerName="console" Jul 23 14:58:46.643870 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643842 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" containerName="registry" Jul 23 14:58:46.643870 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643852 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" containerName="registry" Jul 23 14:58:46.643958 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643927 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8950e9f9-7e54-4502-a46a-366135a2dfc5" containerName="console" Jul 23 14:58:46.643958 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.643940 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4c2533-1907-4fdc-aa02-4dcb7d526917" containerName="registry" Jul 23 14:58:46.647568 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.647545 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.660596 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.660565 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f9596797-w5hdr"] Jul 23 14:58:46.661096 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.661066 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xjwws" event={"ID":"3d60ecea-1743-434c-8b1c-78b027daac9b","Type":"ContainerStarted","Data":"ec3fba46133e33793377d3e1cab41a4d1878eb6ab06e3455f7f82dbb97034e2b"} Jul 23 14:58:46.823461 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78673a50-369a-4b33-8675-21dab537126c-console-serving-cert\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.823631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78673a50-369a-4b33-8675-21dab537126c-console-oauth-config\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.823631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-trusted-ca-bundle\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.823631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823510 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-console-config\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.823631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-oauth-serving-cert\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.823631 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-service-ca\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.823787 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.823636 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp4rb\" (UniqueName: \"kubernetes.io/projected/78673a50-369a-4b33-8675-21dab537126c-kube-api-access-lp4rb\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924445 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78673a50-369a-4b33-8675-21dab537126c-console-oauth-config\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924445 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-trusted-ca-bundle\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-console-config\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-oauth-serving-cert\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924519 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-service-ca\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp4rb\" (UniqueName: \"kubernetes.io/projected/78673a50-369a-4b33-8675-21dab537126c-kube-api-access-lp4rb\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.924659 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.924627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78673a50-369a-4b33-8675-21dab537126c-console-serving-cert\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.925297 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.925223 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-console-config\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.925415 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.925370 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-oauth-serving-cert\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.925415 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.925385 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-service-ca\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.925690 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.925666 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78673a50-369a-4b33-8675-21dab537126c-trusted-ca-bundle\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.927243 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.927218 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78673a50-369a-4b33-8675-21dab537126c-console-oauth-config\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.927348 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.927227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78673a50-369a-4b33-8675-21dab537126c-console-serving-cert\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.933770 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.933751 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp4rb\" (UniqueName: \"kubernetes.io/projected/78673a50-369a-4b33-8675-21dab537126c-kube-api-access-lp4rb\") pod \"console-55f9596797-w5hdr\" (UID: \"78673a50-369a-4b33-8675-21dab537126c\") " pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:46.959584 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:46.959561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:47.103162 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:47.103110 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f9596797-w5hdr"] Jul 23 14:58:47.105323 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:58:47.105298 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78673a50_369a_4b33_8675_21dab537126c.slice/crio-4b8fdac920167cc6bc2e5d768ccb58e5fd2ab26a19337ff101e8149b86f9e582 WatchSource:0}: Error finding container 4b8fdac920167cc6bc2e5d768ccb58e5fd2ab26a19337ff101e8149b86f9e582: Status 404 returned error can't find the container with id 4b8fdac920167cc6bc2e5d768ccb58e5fd2ab26a19337ff101e8149b86f9e582 Jul 23 14:58:47.668173 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:47.668135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f9596797-w5hdr" event={"ID":"78673a50-369a-4b33-8675-21dab537126c","Type":"ContainerStarted","Data":"bc1278f4b8f3c31433ebba01c866caf74436de3ee305a37484bc33ee4382502b"} Jul 23 14:58:47.668590 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:47.668197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f9596797-w5hdr" event={"ID":"78673a50-369a-4b33-8675-21dab537126c","Type":"ContainerStarted","Data":"4b8fdac920167cc6bc2e5d768ccb58e5fd2ab26a19337ff101e8149b86f9e582"} Jul 23 14:58:47.686891 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:47.686851 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55f9596797-w5hdr" podStartSLOduration=1.686832814 podStartE2EDuration="1.686832814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:58:47.685610661 +0000 UTC m=+375.837060030" watchObservedRunningTime="2026-07-23 14:58:47.686832814 +0000 UTC m=+375.838282184" Jul 23 14:58:48.674217 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:48.674179 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xjwws" event={"ID":"3d60ecea-1743-434c-8b1c-78b027daac9b","Type":"ContainerStarted","Data":"1cd30f235334577ef63ffe937e8eca24b2548527b3d3a08f67e5404ed5c7b455"} Jul 23 14:58:48.674626 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:48.674243 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xjwws" event={"ID":"3d60ecea-1743-434c-8b1c-78b027daac9b","Type":"ContainerStarted","Data":"fb44e488e2f131884b3b2cfe2097bdcf87915d9d674c7beb3431319d34dc883a"} Jul 23 14:58:48.692733 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:48.692690 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xjwws" podStartSLOduration=375.704018243 podStartE2EDuration="6m16.692675192s" totalImagesPullingTime="988.656949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:58:48.691466378 +0000 UTC m=+376.842915747" watchObservedRunningTime="2026-07-23 14:58:48.692675192 +0000 UTC m=+376.844124558" Jul 23 14:58:56.959974 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:56.959937 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:56.960449 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:56.960110 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:56.965050 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:56.965023 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:57.710915 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:57.710883 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55f9596797-w5hdr" Jul 23 14:58:57.759061 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:58:57.759031 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5984cc87c-fd775"] Jul 23 14:59:22.783935 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:22.783871 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5984cc87c-fd775" podUID="c741be1b-d719-4648-93f0-29355a2c5e4b" containerName="console" containerID="cri-o://9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1" gracePeriod=15 Jul 23 14:59:23.027090 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.027067 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:59:23.117032 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.116941 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-service-ca\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117032 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.116996 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-serving-cert\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117032 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117015 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-trusted-ca-bundle\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117336 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117040 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v77sk\" (UniqueName: \"kubernetes.io/projected/c741be1b-d719-4648-93f0-29355a2c5e4b-kube-api-access-v77sk\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117336 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117190 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-console-config\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117336 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117290 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-oauth-config\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117336 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117320 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-oauth-serving-cert\") pod \"c741be1b-d719-4648-93f0-29355a2c5e4b\" (UID: \"c741be1b-d719-4648-93f0-29355a2c5e4b\") " Jul 23 14:59:23.117543 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117426 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-service-ca" (OuterVolumeSpecName: "service-ca") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:59:23.117543 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117442 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:59:23.117637 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117554 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.117637 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117571 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-trusted-ca-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.117749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117625 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-console-config" (OuterVolumeSpecName: "console-config") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:59:23.117998 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.117973 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:59:23.119442 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.119418 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c741be1b-d719-4648-93f0-29355a2c5e4b-kube-api-access-v77sk" (OuterVolumeSpecName: "kube-api-access-v77sk") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "kube-api-access-v77sk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:59:23.119442 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.119419 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:59:23.119442 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.119432 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c741be1b-d719-4648-93f0-29355a2c5e4b" (UID: "c741be1b-d719-4648-93f0-29355a2c5e4b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:59:23.218547 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.218509 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-oauth-config\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.218547 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.218550 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-oauth-serving-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.218749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.218565 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c741be1b-d719-4648-93f0-29355a2c5e4b-console-serving-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.218749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.218580 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v77sk\" (UniqueName: \"kubernetes.io/projected/c741be1b-d719-4648-93f0-29355a2c5e4b-kube-api-access-v77sk\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.218749 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.218594 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c741be1b-d719-4648-93f0-29355a2c5e4b-console-config\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 14:59:23.803894 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.803859 2582 generic.go:356] "Generic (PLEG): container finished" podID="c741be1b-d719-4648-93f0-29355a2c5e4b" containerID="9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1" exitCode=0 Jul 23 14:59:23.804324 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.803921 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5984cc87c-fd775" Jul 23 14:59:23.804324 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.803930 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5984cc87c-fd775" event={"ID":"c741be1b-d719-4648-93f0-29355a2c5e4b","Type":"ContainerDied","Data":"9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1"} Jul 23 14:59:23.804324 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.803981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5984cc87c-fd775" event={"ID":"c741be1b-d719-4648-93f0-29355a2c5e4b","Type":"ContainerDied","Data":"ad77fa67f9a1f78267aa9aaa9bc55472c13b9c247bb9fe03709cf41285016e1e"} Jul 23 14:59:23.804324 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.804008 2582 scope.go:117] "RemoveContainer" containerID="9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1" Jul 23 14:59:23.812854 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.812838 2582 scope.go:117] "RemoveContainer" containerID="9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1" Jul 23 14:59:23.813094 ip-10-0-142-11 kubenswrapper[2582]: E0723 14:59:23.813073 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1\": container with ID starting with 9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1 not found: ID does not exist" containerID="9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1" Jul 23 14:59:23.813161 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.813103 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1"} err="failed to get container status \"9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1\": rpc error: code = NotFound desc = could not find container \"9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1\": container with ID starting with 9ef4f456264698e89c74359aa9ac227699e687d882a498acd610839f7a35edc1 not found: ID does not exist" Jul 23 14:59:23.824364 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.824339 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5984cc87c-fd775"] Jul 23 14:59:23.827681 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:23.827659 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5984cc87c-fd775"] Jul 23 14:59:24.293925 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:24.293893 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c741be1b-d719-4648-93f0-29355a2c5e4b" path="/var/lib/kubelet/pods/c741be1b-d719-4648-93f0-29355a2c5e4b/volumes" Jul 23 14:59:51.852953 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.852918 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl"] Jul 23 14:59:51.853348 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.853230 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c741be1b-d719-4648-93f0-29355a2c5e4b" containerName="console" Jul 23 14:59:51.853348 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.853240 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c741be1b-d719-4648-93f0-29355a2c5e4b" containerName="console" Jul 23 14:59:51.853348 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.853347 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c741be1b-d719-4648-93f0-29355a2c5e4b" containerName="console" Jul 23 14:59:51.857996 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.857980 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:51.861024 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.861001 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 14:59:51.861174 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.861042 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 14:59:51.862308 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.862292 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s82dh\"" Jul 23 14:59:51.865178 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.865154 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl"] Jul 23 14:59:51.956073 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.956036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktjxn\" (UniqueName: \"kubernetes.io/projected/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-kube-api-access-ktjxn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:51.956317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.956090 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:51.956317 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:51.956211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.057344 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.057300 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.057344 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.057352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktjxn\" (UniqueName: \"kubernetes.io/projected/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-kube-api-access-ktjxn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.057594 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.057386 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.057803 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.057779 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.057803 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.057796 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.066477 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.066454 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktjxn\" (UniqueName: \"kubernetes.io/projected/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-kube-api-access-ktjxn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.167605 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.167580 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 14:59:52.298315 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.298288 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl"] Jul 23 14:59:52.301596 ip-10-0-142-11 kubenswrapper[2582]: W0723 14:59:52.301564 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8b0794_4ea7_447d_b4d6_a3fac3b5d021.slice/crio-2542658391ecc7982c152a7850ec4568d60d682165f87d2d6ade7a2861973e12 WatchSource:0}: Error finding container 2542658391ecc7982c152a7850ec4568d60d682165f87d2d6ade7a2861973e12: Status 404 returned error can't find the container with id 2542658391ecc7982c152a7850ec4568d60d682165f87d2d6ade7a2861973e12 Jul 23 14:59:52.914505 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:52.914463 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" event={"ID":"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021","Type":"ContainerStarted","Data":"2542658391ecc7982c152a7850ec4568d60d682165f87d2d6ade7a2861973e12"} Jul 23 14:59:58.940940 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:58.940904 2582 generic.go:356] "Generic (PLEG): container finished" podID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerID="d76494e48cbe2d3dce44ae1594eee8b8c6e6d04b64906b202149cc13126ee97d" exitCode=0 Jul 23 14:59:58.941430 ip-10-0-142-11 kubenswrapper[2582]: I0723 14:59:58.940972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" event={"ID":"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021","Type":"ContainerDied","Data":"d76494e48cbe2d3dce44ae1594eee8b8c6e6d04b64906b202149cc13126ee97d"} Jul 23 15:00:01.955585 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:01.955548 2582 generic.go:356] "Generic (PLEG): container finished" podID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerID="510de1c3a9259d7460b7f86007db1b0053c601a72a60ea06116d170704d22093" exitCode=0 Jul 23 15:00:01.956051 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:01.955640 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" event={"ID":"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021","Type":"ContainerDied","Data":"510de1c3a9259d7460b7f86007db1b0053c601a72a60ea06116d170704d22093"} Jul 23 15:00:10.994880 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:10.994842 2582 generic.go:356] "Generic (PLEG): container finished" podID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerID="e152fe0410fe8d66e5e23eb2626c6ed36e366ec6763fc7109cf12c8fe04449ad" exitCode=0 Jul 23 15:00:10.995323 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:10.994930 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" event={"ID":"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021","Type":"ContainerDied","Data":"e152fe0410fe8d66e5e23eb2626c6ed36e366ec6763fc7109cf12c8fe04449ad"} Jul 23 15:00:12.124890 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.124868 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 15:00:12.226461 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.226431 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktjxn\" (UniqueName: \"kubernetes.io/projected/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-kube-api-access-ktjxn\") pod \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " Jul 23 15:00:12.226624 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.226504 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-bundle\") pod \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " Jul 23 15:00:12.226624 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.226550 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-util\") pod \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\" (UID: \"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021\") " Jul 23 15:00:12.227157 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.227125 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-bundle" (OuterVolumeSpecName: "bundle") pod "8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" (UID: "8e8b0794-4ea7-447d-b4d6-a3fac3b5d021"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:00:12.228816 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.228796 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-kube-api-access-ktjxn" (OuterVolumeSpecName: "kube-api-access-ktjxn") pod "8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" (UID: "8e8b0794-4ea7-447d-b4d6-a3fac3b5d021"). InnerVolumeSpecName "kube-api-access-ktjxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:00:12.230718 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.230696 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-util" (OuterVolumeSpecName: "util") pod "8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" (UID: "8e8b0794-4ea7-447d-b4d6-a3fac3b5d021"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:00:12.327535 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.327503 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:12.327535 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.327526 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:12.327535 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:12.327535 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktjxn\" (UniqueName: \"kubernetes.io/projected/8e8b0794-4ea7-447d-b4d6-a3fac3b5d021-kube-api-access-ktjxn\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:13.004207 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:13.004177 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" Jul 23 15:00:13.004424 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:13.004172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwqsl" event={"ID":"8e8b0794-4ea7-447d-b4d6-a3fac3b5d021","Type":"ContainerDied","Data":"2542658391ecc7982c152a7850ec4568d60d682165f87d2d6ade7a2861973e12"} Jul 23 15:00:13.004424 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:13.004308 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2542658391ecc7982c152a7850ec4568d60d682165f87d2d6ade7a2861973e12" Jul 23 15:00:19.930331 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930294 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz"] Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930594 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="util" Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930604 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="util" Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930612 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="pull" Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930617 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="pull" Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930631 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="extract" Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930652 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="extract" Jul 23 15:00:19.930717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.930698 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e8b0794-4ea7-447d-b4d6-a3fac3b5d021" containerName="extract" Jul 23 15:00:19.954069 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.954040 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz"] Jul 23 15:00:19.954235 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.954132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:19.957054 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.957030 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:00:19.957054 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.957048 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 15:00:19.957240 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.957128 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-p2npv\"" Jul 23 15:00:19.994011 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.993980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5pvm\" (UniqueName: \"kubernetes.io/projected/37500c52-7eff-4045-a862-9df250eec93b-kube-api-access-f5pvm\") pod \"cert-manager-operator-controller-manager-55c444bf86-sztnz\" (UID: \"37500c52-7eff-4045-a862-9df250eec93b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:19.994139 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:19.994045 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37500c52-7eff-4045-a862-9df250eec93b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sztnz\" (UID: \"37500c52-7eff-4045-a862-9df250eec93b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:20.094750 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:20.094717 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5pvm\" (UniqueName: \"kubernetes.io/projected/37500c52-7eff-4045-a862-9df250eec93b-kube-api-access-f5pvm\") pod \"cert-manager-operator-controller-manager-55c444bf86-sztnz\" (UID: \"37500c52-7eff-4045-a862-9df250eec93b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:20.094932 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:20.094770 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37500c52-7eff-4045-a862-9df250eec93b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sztnz\" (UID: \"37500c52-7eff-4045-a862-9df250eec93b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:20.095083 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:20.095069 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37500c52-7eff-4045-a862-9df250eec93b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sztnz\" (UID: \"37500c52-7eff-4045-a862-9df250eec93b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:20.106575 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:20.106545 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5pvm\" (UniqueName: \"kubernetes.io/projected/37500c52-7eff-4045-a862-9df250eec93b-kube-api-access-f5pvm\") pod \"cert-manager-operator-controller-manager-55c444bf86-sztnz\" (UID: \"37500c52-7eff-4045-a862-9df250eec93b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:20.263872 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:20.263793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" Jul 23 15:00:20.419940 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:20.419910 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz"] Jul 23 15:00:20.423190 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:00:20.423165 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37500c52_7eff_4045_a862_9df250eec93b.slice/crio-cae6832e489d267a99e3b4ce2ced1ef23fe5b534492e25c33cb41dcdbdfcff56 WatchSource:0}: Error finding container cae6832e489d267a99e3b4ce2ced1ef23fe5b534492e25c33cb41dcdbdfcff56: Status 404 returned error can't find the container with id cae6832e489d267a99e3b4ce2ced1ef23fe5b534492e25c33cb41dcdbdfcff56 Jul 23 15:00:21.041966 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:21.041932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" event={"ID":"37500c52-7eff-4045-a862-9df250eec93b","Type":"ContainerStarted","Data":"cae6832e489d267a99e3b4ce2ced1ef23fe5b534492e25c33cb41dcdbdfcff56"} Jul 23 15:00:23.052470 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.052433 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" event={"ID":"37500c52-7eff-4045-a862-9df250eec93b","Type":"ContainerStarted","Data":"18ecf341385cf8b62e56c1521b4e6bcf7f5eb4572c5aeef62ae99e00d6d39cd3"} Jul 23 15:00:23.077851 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.077799 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sztnz" podStartSLOduration=2.462248372 podStartE2EDuration="4.077782806s" totalImagesPullingTime="1.615534434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:00:23.077395176 +0000 UTC m=+471.228844549" watchObservedRunningTime="2026-07-23 15:00:23.077782806 +0000 UTC m=+471.229232174" Jul 23 15:00:23.859344 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.859309 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7"] Jul 23 15:00:23.861602 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.861584 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:23.864848 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.864832 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 15:00:23.865119 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.865106 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 15:00:23.866196 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.866170 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s82dh\"" Jul 23 15:00:23.884135 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.884114 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7"] Jul 23 15:00:23.930056 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.930027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:23.930189 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.930068 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtjl\" (UniqueName: \"kubernetes.io/projected/7ab243bf-1e09-48fb-af19-9052047f032b-kube-api-access-wbtjl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:23.930189 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:23.930152 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.031310 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.031280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtjl\" (UniqueName: \"kubernetes.io/projected/7ab243bf-1e09-48fb-af19-9052047f032b-kube-api-access-wbtjl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.031504 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.031344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.031504 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.031385 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.031676 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.031656 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.031763 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.031742 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.047695 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.047670 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtjl\" (UniqueName: \"kubernetes.io/projected/7ab243bf-1e09-48fb-af19-9052047f032b-kube-api-access-wbtjl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.170865 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.170835 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:24.301115 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:24.301084 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7"] Jul 23 15:00:24.303351 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:00:24.303323 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab243bf_1e09_48fb_af19_9052047f032b.slice/crio-e813ad0b1882cd21cd0d6a290edddd59f307d19733166fb62c6aa4c234829a6b WatchSource:0}: Error finding container e813ad0b1882cd21cd0d6a290edddd59f307d19733166fb62c6aa4c234829a6b: Status 404 returned error can't find the container with id e813ad0b1882cd21cd0d6a290edddd59f307d19733166fb62c6aa4c234829a6b Jul 23 15:00:25.062413 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:25.062374 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ab243bf-1e09-48fb-af19-9052047f032b" containerID="c6fd48c1aefe623a759a6970406d4052f31a03f8c913d8f7be8e0452eb31f01e" exitCode=0 Jul 23 15:00:25.062544 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:25.062447 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" event={"ID":"7ab243bf-1e09-48fb-af19-9052047f032b","Type":"ContainerDied","Data":"c6fd48c1aefe623a759a6970406d4052f31a03f8c913d8f7be8e0452eb31f01e"} Jul 23 15:00:25.062544 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:25.062496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" event={"ID":"7ab243bf-1e09-48fb-af19-9052047f032b","Type":"ContainerStarted","Data":"e813ad0b1882cd21cd0d6a290edddd59f307d19733166fb62c6aa4c234829a6b"} Jul 23 15:00:27.074600 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.074570 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" event={"ID":"7ab243bf-1e09-48fb-af19-9052047f032b","Type":"ContainerStarted","Data":"c7be610628819b274da46bbc9d2eadc15a890e122b0ec776b697e2b312eee5a0"} Jul 23 15:00:27.145061 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.145028 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc"] Jul 23 15:00:27.149791 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.149775 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.153373 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.153352 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 15:00:27.153604 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.153589 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fr4d4\"" Jul 23 15:00:27.153828 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.153807 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 15:00:27.160378 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.160355 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc"] Jul 23 15:00:27.259741 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.259644 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n95x\" (UniqueName: \"kubernetes.io/projected/50bf3395-a897-442b-a7ae-e1e8b1d370d2-kube-api-access-8n95x\") pod \"cert-manager-cainjector-559cdd8c87-9g2kc\" (UID: \"50bf3395-a897-442b-a7ae-e1e8b1d370d2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.259741 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.259686 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50bf3395-a897-442b-a7ae-e1e8b1d370d2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9g2kc\" (UID: \"50bf3395-a897-442b-a7ae-e1e8b1d370d2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.360750 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.360713 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n95x\" (UniqueName: \"kubernetes.io/projected/50bf3395-a897-442b-a7ae-e1e8b1d370d2-kube-api-access-8n95x\") pod \"cert-manager-cainjector-559cdd8c87-9g2kc\" (UID: \"50bf3395-a897-442b-a7ae-e1e8b1d370d2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.360750 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.360757 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50bf3395-a897-442b-a7ae-e1e8b1d370d2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9g2kc\" (UID: \"50bf3395-a897-442b-a7ae-e1e8b1d370d2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.371661 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.371635 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50bf3395-a897-442b-a7ae-e1e8b1d370d2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9g2kc\" (UID: \"50bf3395-a897-442b-a7ae-e1e8b1d370d2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.371905 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.371884 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n95x\" (UniqueName: \"kubernetes.io/projected/50bf3395-a897-442b-a7ae-e1e8b1d370d2-kube-api-access-8n95x\") pod \"cert-manager-cainjector-559cdd8c87-9g2kc\" (UID: \"50bf3395-a897-442b-a7ae-e1e8b1d370d2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.472177 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.472145 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" Jul 23 15:00:27.604363 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:27.604340 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc"] Jul 23 15:00:27.606459 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:00:27.606428 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50bf3395_a897_442b_a7ae_e1e8b1d370d2.slice/crio-87963619dfdbcf132ee3c65653a04f348d5e17c080c8dd7a67bb20d30ce765a7 WatchSource:0}: Error finding container 87963619dfdbcf132ee3c65653a04f348d5e17c080c8dd7a67bb20d30ce765a7: Status 404 returned error can't find the container with id 87963619dfdbcf132ee3c65653a04f348d5e17c080c8dd7a67bb20d30ce765a7 Jul 23 15:00:28.080127 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:28.080089 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" event={"ID":"50bf3395-a897-442b-a7ae-e1e8b1d370d2","Type":"ContainerStarted","Data":"87963619dfdbcf132ee3c65653a04f348d5e17c080c8dd7a67bb20d30ce765a7"} Jul 23 15:00:28.081867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:28.081839 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ab243bf-1e09-48fb-af19-9052047f032b" containerID="c7be610628819b274da46bbc9d2eadc15a890e122b0ec776b697e2b312eee5a0" exitCode=0 Jul 23 15:00:28.081983 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:28.081906 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" event={"ID":"7ab243bf-1e09-48fb-af19-9052047f032b","Type":"ContainerDied","Data":"c7be610628819b274da46bbc9d2eadc15a890e122b0ec776b697e2b312eee5a0"} Jul 23 15:00:29.089061 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:29.089010 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ab243bf-1e09-48fb-af19-9052047f032b" containerID="71b3a0d424a8da85940be7d1bbabf4fef73f7f3e26fe5e44e52f442a9c9795b2" exitCode=0 Jul 23 15:00:29.089512 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:29.089057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" event={"ID":"7ab243bf-1e09-48fb-af19-9052047f032b","Type":"ContainerDied","Data":"71b3a0d424a8da85940be7d1bbabf4fef73f7f3e26fe5e44e52f442a9c9795b2"} Jul 23 15:00:30.246182 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.246163 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:30.389051 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.388963 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-util\") pod \"7ab243bf-1e09-48fb-af19-9052047f032b\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " Jul 23 15:00:30.389051 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.389026 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbtjl\" (UniqueName: \"kubernetes.io/projected/7ab243bf-1e09-48fb-af19-9052047f032b-kube-api-access-wbtjl\") pod \"7ab243bf-1e09-48fb-af19-9052047f032b\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " Jul 23 15:00:30.389281 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.389085 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-bundle\") pod \"7ab243bf-1e09-48fb-af19-9052047f032b\" (UID: \"7ab243bf-1e09-48fb-af19-9052047f032b\") " Jul 23 15:00:30.389504 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.389481 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-bundle" (OuterVolumeSpecName: "bundle") pod "7ab243bf-1e09-48fb-af19-9052047f032b" (UID: "7ab243bf-1e09-48fb-af19-9052047f032b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:00:30.391422 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.391393 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab243bf-1e09-48fb-af19-9052047f032b-kube-api-access-wbtjl" (OuterVolumeSpecName: "kube-api-access-wbtjl") pod "7ab243bf-1e09-48fb-af19-9052047f032b" (UID: "7ab243bf-1e09-48fb-af19-9052047f032b"). InnerVolumeSpecName "kube-api-access-wbtjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:00:30.393619 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.393596 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-util" (OuterVolumeSpecName: "util") pod "7ab243bf-1e09-48fb-af19-9052047f032b" (UID: "7ab243bf-1e09-48fb-af19-9052047f032b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:00:30.490023 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.489986 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:30.490023 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.490017 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbtjl\" (UniqueName: \"kubernetes.io/projected/7ab243bf-1e09-48fb-af19-9052047f032b-kube-api-access-wbtjl\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:30.490023 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:30.490026 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ab243bf-1e09-48fb-af19-9052047f032b-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:31.105806 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.105773 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" event={"ID":"50bf3395-a897-442b-a7ae-e1e8b1d370d2","Type":"ContainerStarted","Data":"dd7874a6b189741b3c9998f5b50e717f50d915690c66ef8fc0b353bd14468fd9"} Jul 23 15:00:31.107559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.107538 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" Jul 23 15:00:31.107559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.107541 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj2pq7" event={"ID":"7ab243bf-1e09-48fb-af19-9052047f032b","Type":"ContainerDied","Data":"e813ad0b1882cd21cd0d6a290edddd59f307d19733166fb62c6aa4c234829a6b"} Jul 23 15:00:31.107705 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.107607 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e813ad0b1882cd21cd0d6a290edddd59f307d19733166fb62c6aa4c234829a6b" Jul 23 15:00:31.125671 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.125625 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9g2kc" podStartSLOduration=1.549432044 podStartE2EDuration="4.125612427s" totalImagesPullingTime="2.576180383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:00:31.125126252 +0000 UTC m=+479.276575623" watchObservedRunningTime="2026-07-23 15:00:31.125612427 +0000 UTC m=+479.277061796" Jul 23 15:00:31.150604 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150579 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sw8kw"] Jul 23 15:00:31.150882 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150858 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="extract" Jul 23 15:00:31.150882 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150873 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="extract" Jul 23 15:00:31.151060 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150891 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="pull" Jul 23 15:00:31.151060 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150898 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="pull" Jul 23 15:00:31.151060 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150910 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="util" Jul 23 15:00:31.151060 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150917 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="util" Jul 23 15:00:31.151060 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.150999 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab243bf-1e09-48fb-af19-9052047f032b" containerName="extract" Jul 23 15:00:31.153057 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.153038 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.156023 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.156006 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7j9wr\"" Jul 23 15:00:31.165099 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.165078 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sw8kw"] Jul 23 15:00:31.297180 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.297145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0f386c6-b6e4-4264-8bb8-ed398c45cdc8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sw8kw\" (UID: \"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.297596 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.297210 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7djz\" (UniqueName: \"kubernetes.io/projected/c0f386c6-b6e4-4264-8bb8-ed398c45cdc8-kube-api-access-w7djz\") pod \"cert-manager-webhook-6486ffd997-sw8kw\" (UID: \"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.398397 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.398297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0f386c6-b6e4-4264-8bb8-ed398c45cdc8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sw8kw\" (UID: \"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.398570 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.398392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7djz\" (UniqueName: \"kubernetes.io/projected/c0f386c6-b6e4-4264-8bb8-ed398c45cdc8-kube-api-access-w7djz\") pod \"cert-manager-webhook-6486ffd997-sw8kw\" (UID: \"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.407779 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.407755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0f386c6-b6e4-4264-8bb8-ed398c45cdc8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sw8kw\" (UID: \"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.408041 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.408019 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7djz\" (UniqueName: \"kubernetes.io/projected/c0f386c6-b6e4-4264-8bb8-ed398c45cdc8-kube-api-access-w7djz\") pod \"cert-manager-webhook-6486ffd997-sw8kw\" (UID: \"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.462462 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.462422 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:31.594730 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:31.594707 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sw8kw"] Jul 23 15:00:31.596405 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:00:31.596380 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0f386c6_b6e4_4264_8bb8_ed398c45cdc8.slice/crio-34edcad9fd2f3b78ae329a7a923eb173b8f9c4c4cf113a039b007e8457d152b1 WatchSource:0}: Error finding container 34edcad9fd2f3b78ae329a7a923eb173b8f9c4c4cf113a039b007e8457d152b1: Status 404 returned error can't find the container with id 34edcad9fd2f3b78ae329a7a923eb173b8f9c4c4cf113a039b007e8457d152b1 Jul 23 15:00:32.113731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:32.113694 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" event={"ID":"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8","Type":"ContainerStarted","Data":"95b4c05ac68b80647ca8412e9a73a0040cba232af894d9a70cdd970186bc2616"} Jul 23 15:00:32.113731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:32.113741 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" event={"ID":"c0f386c6-b6e4-4264-8bb8-ed398c45cdc8","Type":"ContainerStarted","Data":"34edcad9fd2f3b78ae329a7a923eb173b8f9c4c4cf113a039b007e8457d152b1"} Jul 23 15:00:32.113950 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:32.113762 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:32.132724 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:32.132669 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" podStartSLOduration=1.132654664 podStartE2EDuration="1.132654664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:00:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:00:32.132158291 +0000 UTC m=+480.283607661" watchObservedRunningTime="2026-07-23 15:00:32.132654664 +0000 UTC m=+480.284104033" Jul 23 15:00:35.206408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.206374 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpl84"] Jul 23 15:00:35.208667 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.208651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.212398 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.212379 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-s5tlx\"" Jul 23 15:00:35.223804 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.223780 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpl84"] Jul 23 15:00:35.334230 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.334195 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8g5\" (UniqueName: \"kubernetes.io/projected/cb00aeb5-53e9-4a97-81e7-c313c923e1de-kube-api-access-5b8g5\") pod \"cert-manager-55b4d954d5-kpl84\" (UID: \"cb00aeb5-53e9-4a97-81e7-c313c923e1de\") " pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.334429 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.334245 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb00aeb5-53e9-4a97-81e7-c313c923e1de-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpl84\" (UID: \"cb00aeb5-53e9-4a97-81e7-c313c923e1de\") " pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.435599 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.435568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8g5\" (UniqueName: \"kubernetes.io/projected/cb00aeb5-53e9-4a97-81e7-c313c923e1de-kube-api-access-5b8g5\") pod \"cert-manager-55b4d954d5-kpl84\" (UID: \"cb00aeb5-53e9-4a97-81e7-c313c923e1de\") " pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.435783 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.435626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb00aeb5-53e9-4a97-81e7-c313c923e1de-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpl84\" (UID: \"cb00aeb5-53e9-4a97-81e7-c313c923e1de\") " pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.465884 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.465810 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8g5\" (UniqueName: \"kubernetes.io/projected/cb00aeb5-53e9-4a97-81e7-c313c923e1de-kube-api-access-5b8g5\") pod \"cert-manager-55b4d954d5-kpl84\" (UID: \"cb00aeb5-53e9-4a97-81e7-c313c923e1de\") " pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.465884 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.465830 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb00aeb5-53e9-4a97-81e7-c313c923e1de-bound-sa-token\") pod \"cert-manager-55b4d954d5-kpl84\" (UID: \"cb00aeb5-53e9-4a97-81e7-c313c923e1de\") " pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.518041 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.518004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kpl84" Jul 23 15:00:35.668559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:35.668537 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kpl84"] Jul 23 15:00:35.670088 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:00:35.670064 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb00aeb5_53e9_4a97_81e7_c313c923e1de.slice/crio-5eb0a02098798ba79c55e1698e2a3bb862b1569060eff8957bb36126d18ae91f WatchSource:0}: Error finding container 5eb0a02098798ba79c55e1698e2a3bb862b1569060eff8957bb36126d18ae91f: Status 404 returned error can't find the container with id 5eb0a02098798ba79c55e1698e2a3bb862b1569060eff8957bb36126d18ae91f Jul 23 15:00:36.132786 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:36.132689 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kpl84" event={"ID":"cb00aeb5-53e9-4a97-81e7-c313c923e1de","Type":"ContainerStarted","Data":"c2e7b5804926a2f09d13ffc3f0b39400b4bd2bafc0260e172929adff9305697d"} Jul 23 15:00:36.132786 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:36.132739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kpl84" event={"ID":"cb00aeb5-53e9-4a97-81e7-c313c923e1de","Type":"ContainerStarted","Data":"5eb0a02098798ba79c55e1698e2a3bb862b1569060eff8957bb36126d18ae91f"} Jul 23 15:00:36.159320 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:36.159242 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-kpl84" podStartSLOduration=1.159227947 podStartE2EDuration="1.159227947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:00:36.157109391 +0000 UTC m=+484.308558759" watchObservedRunningTime="2026-07-23 15:00:36.159227947 +0000 UTC m=+484.310677316" Jul 23 15:00:38.121213 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:38.121182 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sw8kw" Jul 23 15:00:43.832412 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:43.832376 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf"] Jul 23 15:00:43.834840 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:43.834824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:43.837893 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:43.837872 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 15:00:43.838004 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:43.837901 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s82dh\"" Jul 23 15:00:43.839307 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:43.839287 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 15:00:43.847164 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:43.847143 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf"] Jul 23 15:00:44.013298 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.013249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8q6r\" (UniqueName: \"kubernetes.io/projected/70207f0d-6892-4048-b701-26fc347c5998-kube-api-access-r8q6r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.013466 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.013345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.013466 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.013423 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.114408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.114317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.114408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.114409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q6r\" (UniqueName: \"kubernetes.io/projected/70207f0d-6892-4048-b701-26fc347c5998-kube-api-access-r8q6r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.114596 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.114440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.114679 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.114606 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.114714 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.114705 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.124601 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.124572 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8q6r\" (UniqueName: \"kubernetes.io/projected/70207f0d-6892-4048-b701-26fc347c5998-kube-api-access-r8q6r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.144382 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.144358 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:44.274944 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:44.274922 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf"] Jul 23 15:00:44.277685 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:00:44.277650 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70207f0d_6892_4048_b701_26fc347c5998.slice/crio-a2e54de8ed103cb71a7855fcbd212775b594adee4b18fd1129e0d1da1f3a6aa3 WatchSource:0}: Error finding container a2e54de8ed103cb71a7855fcbd212775b594adee4b18fd1129e0d1da1f3a6aa3: Status 404 returned error can't find the container with id a2e54de8ed103cb71a7855fcbd212775b594adee4b18fd1129e0d1da1f3a6aa3 Jul 23 15:00:45.174849 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:45.174810 2582 generic.go:356] "Generic (PLEG): container finished" podID="70207f0d-6892-4048-b701-26fc347c5998" containerID="ae5cfab1d22cba39feb083362d25351a9be4cbd5f3a77d6b01e122cc1cd300ce" exitCode=0 Jul 23 15:00:45.175331 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:45.175271 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" event={"ID":"70207f0d-6892-4048-b701-26fc347c5998","Type":"ContainerDied","Data":"ae5cfab1d22cba39feb083362d25351a9be4cbd5f3a77d6b01e122cc1cd300ce"} Jul 23 15:00:45.175408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:45.175340 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" event={"ID":"70207f0d-6892-4048-b701-26fc347c5998","Type":"ContainerStarted","Data":"a2e54de8ed103cb71a7855fcbd212775b594adee4b18fd1129e0d1da1f3a6aa3"} Jul 23 15:00:46.182661 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:46.182631 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" event={"ID":"70207f0d-6892-4048-b701-26fc347c5998","Type":"ContainerStarted","Data":"b886e3e6360eb551a1e94110e1c2d2dee3320d7892b5699e06bd4eae00bfd4d4"} Jul 23 15:00:47.188708 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:47.188678 2582 generic.go:356] "Generic (PLEG): container finished" podID="70207f0d-6892-4048-b701-26fc347c5998" containerID="b886e3e6360eb551a1e94110e1c2d2dee3320d7892b5699e06bd4eae00bfd4d4" exitCode=0 Jul 23 15:00:47.189084 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:47.188745 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" event={"ID":"70207f0d-6892-4048-b701-26fc347c5998","Type":"ContainerDied","Data":"b886e3e6360eb551a1e94110e1c2d2dee3320d7892b5699e06bd4eae00bfd4d4"} Jul 23 15:00:48.195684 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:48.195652 2582 generic.go:356] "Generic (PLEG): container finished" podID="70207f0d-6892-4048-b701-26fc347c5998" containerID="905ad3acf96a6938c3dac7d20a1ce582dfbf34d80923a2688ec40d362ae5b550" exitCode=0 Jul 23 15:00:48.196058 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:48.195753 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" event={"ID":"70207f0d-6892-4048-b701-26fc347c5998","Type":"ContainerDied","Data":"905ad3acf96a6938c3dac7d20a1ce582dfbf34d80923a2688ec40d362ae5b550"} Jul 23 15:00:49.325401 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.325376 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:49.360563 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.360534 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-bundle\") pod \"70207f0d-6892-4048-b701-26fc347c5998\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " Jul 23 15:00:49.360733 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.360595 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8q6r\" (UniqueName: \"kubernetes.io/projected/70207f0d-6892-4048-b701-26fc347c5998-kube-api-access-r8q6r\") pod \"70207f0d-6892-4048-b701-26fc347c5998\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " Jul 23 15:00:49.360733 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.360627 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-util\") pod \"70207f0d-6892-4048-b701-26fc347c5998\" (UID: \"70207f0d-6892-4048-b701-26fc347c5998\") " Jul 23 15:00:49.361439 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.361411 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-bundle" (OuterVolumeSpecName: "bundle") pod "70207f0d-6892-4048-b701-26fc347c5998" (UID: "70207f0d-6892-4048-b701-26fc347c5998"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:00:49.362824 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.362799 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70207f0d-6892-4048-b701-26fc347c5998-kube-api-access-r8q6r" (OuterVolumeSpecName: "kube-api-access-r8q6r") pod "70207f0d-6892-4048-b701-26fc347c5998" (UID: "70207f0d-6892-4048-b701-26fc347c5998"). InnerVolumeSpecName "kube-api-access-r8q6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:00:49.366489 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.366467 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-util" (OuterVolumeSpecName: "util") pod "70207f0d-6892-4048-b701-26fc347c5998" (UID: "70207f0d-6892-4048-b701-26fc347c5998"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:00:49.461361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.461245 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8q6r\" (UniqueName: \"kubernetes.io/projected/70207f0d-6892-4048-b701-26fc347c5998-kube-api-access-r8q6r\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:49.461361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.461305 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:49.461361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:49.461315 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70207f0d-6892-4048-b701-26fc347c5998-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:00:50.212644 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:50.212612 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" Jul 23 15:00:50.212814 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:50.212612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvmwhf" event={"ID":"70207f0d-6892-4048-b701-26fc347c5998","Type":"ContainerDied","Data":"a2e54de8ed103cb71a7855fcbd212775b594adee4b18fd1129e0d1da1f3a6aa3"} Jul 23 15:00:50.212814 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:00:50.212768 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2e54de8ed103cb71a7855fcbd212775b594adee4b18fd1129e0d1da1f3a6aa3" Jul 23 15:01:12.557973 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.557932 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv"] Jul 23 15:01:12.558445 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558344 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="pull" Jul 23 15:01:12.558445 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558359 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="pull" Jul 23 15:01:12.558445 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558380 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="extract" Jul 23 15:01:12.558445 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558390 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="extract" Jul 23 15:01:12.558445 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558403 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="util" Jul 23 15:01:12.558445 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558409 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="util" Jul 23 15:01:12.558633 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.558465 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="70207f0d-6892-4048-b701-26fc347c5998" containerName="extract" Jul 23 15:01:12.562904 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.562886 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.568028 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.568002 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 15:01:12.568172 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.568120 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 15:01:12.568279 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.568238 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 15:01:12.568392 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.568282 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hqkq8\"" Jul 23 15:01:12.568392 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.568317 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 15:01:12.578903 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.578873 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv"] Jul 23 15:01:12.656892 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.656850 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-manifests\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.657080 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.656900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-tmp\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.657080 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.656927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.657080 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.656957 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pdm6\" (UniqueName: \"kubernetes.io/projected/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-kube-api-access-2pdm6\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.657183 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.657117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-webhook-cert\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.758353 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.758316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-webhook-cert\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.758353 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.758360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-manifests\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.758612 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.758383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-tmp\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.758612 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.758404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.758612 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.758429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdm6\" (UniqueName: \"kubernetes.io/projected/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-kube-api-access-2pdm6\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.758806 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.758778 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-manifests\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.760828 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.760807 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-tmp\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.761219 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.761199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.761275 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.761199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-webhook-cert\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.776183 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.776156 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pdm6\" (UniqueName: \"kubernetes.io/projected/ef5575ac-6836-4dda-81dc-e010dfeb5b3e-kube-api-access-2pdm6\") pod \"opendatahub-operator-controller-manager-56b97c94c-wwbwv\" (UID: \"ef5575ac-6836-4dda-81dc-e010dfeb5b3e\") " pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:12.873855 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:12.873762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:13.040481 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:13.040452 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv"] Jul 23 15:01:13.313428 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:13.313391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" event={"ID":"ef5575ac-6836-4dda-81dc-e010dfeb5b3e","Type":"ContainerStarted","Data":"21c7f62cf719c8e20a193a9d7a82305d84cf00b0b0bd5e802f3c9727b41cc6d0"} Jul 23 15:01:16.331911 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:16.331819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" event={"ID":"ef5575ac-6836-4dda-81dc-e010dfeb5b3e","Type":"ContainerStarted","Data":"25e5492d4e7620c3f51f2644ff66f93cd718987428584a3d5e91b5997eeac9ab"} Jul 23 15:01:17.338752 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:17.338717 2582 generic.go:356] "Generic (PLEG): container finished" podID="ef5575ac-6836-4dda-81dc-e010dfeb5b3e" containerID="25e5492d4e7620c3f51f2644ff66f93cd718987428584a3d5e91b5997eeac9ab" exitCode=0 Jul 23 15:01:17.339134 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:17.338771 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" event={"ID":"ef5575ac-6836-4dda-81dc-e010dfeb5b3e","Type":"ContainerDied","Data":"25e5492d4e7620c3f51f2644ff66f93cd718987428584a3d5e91b5997eeac9ab"} Jul 23 15:01:18.345689 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:18.345646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" event={"ID":"ef5575ac-6836-4dda-81dc-e010dfeb5b3e","Type":"ContainerStarted","Data":"f372f5565a0313d5f70a26bbc0d05b8a915f4a9ef35ea8eb2f31805809ecb6a8"} Jul 23 15:01:18.346156 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:18.345737 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:18.372571 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:18.372521 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" podStartSLOduration=1.818192297 podStartE2EDuration="6.372505766s" totalImagesPullingTime="3.509636963s" totalInitContainerRuntime="1.044676506s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:18.371029774 +0000 UTC m=+526.522479143" watchObservedRunningTime="2026-07-23 15:01:18.372505766 +0000 UTC m=+526.523955135" Jul 23 15:01:29.352846 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:29.352768 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56b97c94c-wwbwv" Jul 23 15:01:33.775620 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.775580 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n"] Jul 23 15:01:33.779116 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.779097 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:33.783902 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.783880 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 15:01:33.784755 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.784732 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 15:01:33.785480 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.785460 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m76xf\"" Jul 23 15:01:33.802811 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.802784 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n"] Jul 23 15:01:33.942804 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.942758 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnpc8\" (UniqueName: \"kubernetes.io/projected/5dcc70a2-4867-4e0d-99fb-0005bd689a71-kube-api-access-hnpc8\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:33.942978 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.942842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dcc70a2-4867-4e0d-99fb-0005bd689a71-tls-certs\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:33.942978 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.942918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dcc70a2-4867-4e0d-99fb-0005bd689a71-tmp\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:33.983942 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.983907 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vgvfp"] Jul 23 15:01:33.986303 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.986280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:33.990351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.990324 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 15:01:33.991489 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:33.991470 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cx4kd\"" Jul 23 15:01:34.001485 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.001458 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vgvfp"] Jul 23 15:01:34.044337 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.044226 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnpc8\" (UniqueName: \"kubernetes.io/projected/5dcc70a2-4867-4e0d-99fb-0005bd689a71-kube-api-access-hnpc8\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.044337 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.044289 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dcc70a2-4867-4e0d-99fb-0005bd689a71-tls-certs\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.044561 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.044363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dcc70a2-4867-4e0d-99fb-0005bd689a71-tmp\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.046696 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.046671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dcc70a2-4867-4e0d-99fb-0005bd689a71-tmp\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.046838 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.046819 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dcc70a2-4867-4e0d-99fb-0005bd689a71-tls-certs\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.074925 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.074896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnpc8\" (UniqueName: \"kubernetes.io/projected/5dcc70a2-4867-4e0d-99fb-0005bd689a71-kube-api-access-hnpc8\") pod \"kube-auth-proxy-8d89f64f-zpd4n\" (UID: \"5dcc70a2-4867-4e0d-99fb-0005bd689a71\") " pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.088653 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.088625 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" Jul 23 15:01:34.145523 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.145416 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvfbf\" (UniqueName: \"kubernetes.io/projected/4ce15863-c157-45e4-bc18-c2b2b92df82b-kube-api-access-tvfbf\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.145724 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.145573 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ce15863-c157-45e4-bc18-c2b2b92df82b-cert\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.246964 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.246929 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvfbf\" (UniqueName: \"kubernetes.io/projected/4ce15863-c157-45e4-bc18-c2b2b92df82b-kube-api-access-tvfbf\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.247140 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.246992 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ce15863-c157-45e4-bc18-c2b2b92df82b-cert\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.247140 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:01:34.247106 2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 23 15:01:34.247212 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:01:34.247165 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce15863-c157-45e4-bc18-c2b2b92df82b-cert podName:4ce15863-c157-45e4-bc18-c2b2b92df82b nodeName:}" failed. No retries permitted until 2026-07-23 15:01:34.747148283 +0000 UTC m=+542.898597629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ce15863-c157-45e4-bc18-c2b2b92df82b-cert") pod "odh-model-controller-7998c67dc4-vgvfp" (UID: "4ce15863-c157-45e4-bc18-c2b2b92df82b") : secret "odh-model-controller-webhook-cert" not found Jul 23 15:01:34.255530 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.255501 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n"] Jul 23 15:01:34.257051 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:01:34.257021 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcc70a2_4867_4e0d_99fb_0005bd689a71.slice/crio-627fb42321ae84eead50dd97ab21f9215c5bcd1ff68de7bcd2b358af2bdc9ae6 WatchSource:0}: Error finding container 627fb42321ae84eead50dd97ab21f9215c5bcd1ff68de7bcd2b358af2bdc9ae6: Status 404 returned error can't find the container with id 627fb42321ae84eead50dd97ab21f9215c5bcd1ff68de7bcd2b358af2bdc9ae6 Jul 23 15:01:34.259139 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.259120 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvfbf\" (UniqueName: \"kubernetes.io/projected/4ce15863-c157-45e4-bc18-c2b2b92df82b-kube-api-access-tvfbf\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.426779 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.426738 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" event={"ID":"5dcc70a2-4867-4e0d-99fb-0005bd689a71","Type":"ContainerStarted","Data":"627fb42321ae84eead50dd97ab21f9215c5bcd1ff68de7bcd2b358af2bdc9ae6"} Jul 23 15:01:34.751076 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.750982 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ce15863-c157-45e4-bc18-c2b2b92df82b-cert\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.753737 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.753714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ce15863-c157-45e4-bc18-c2b2b92df82b-cert\") pod \"odh-model-controller-7998c67dc4-vgvfp\" (UID: \"4ce15863-c157-45e4-bc18-c2b2b92df82b\") " pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.898124 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.898086 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:34.958461 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.958306 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c4p9v"] Jul 23 15:01:34.962383 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.962099 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:34.966221 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.966049 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 15:01:34.966221 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.966205 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-64v4r\"" Jul 23 15:01:34.979495 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:34.979421 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c4p9v"] Jul 23 15:01:35.077001 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.076924 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vgvfp"] Jul 23 15:01:35.081433 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:01:35.081396 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce15863_c157_45e4_bc18_c2b2b92df82b.slice/crio-8b6d89cbc57b42d6d6e465bfb38bdc674b28a4cef5449f6806071c470328e75e WatchSource:0}: Error finding container 8b6d89cbc57b42d6d6e465bfb38bdc674b28a4cef5449f6806071c470328e75e: Status 404 returned error can't find the container with id 8b6d89cbc57b42d6d6e465bfb38bdc674b28a4cef5449f6806071c470328e75e Jul 23 15:01:35.155692 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.155654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288c6c1-8f5a-4c86-98b2-96c54c49dda0-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4p9v\" (UID: \"3288c6c1-8f5a-4c86-98b2-96c54c49dda0\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.155863 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.155745 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46k7k\" (UniqueName: \"kubernetes.io/projected/3288c6c1-8f5a-4c86-98b2-96c54c49dda0-kube-api-access-46k7k\") pod \"model-serving-api-54d89bd5fc-c4p9v\" (UID: \"3288c6c1-8f5a-4c86-98b2-96c54c49dda0\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.180186 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.180146 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vgt49"] Jul 23 15:01:35.183579 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.183555 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.187518 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.186511 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 15:01:35.187518 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.186528 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 15:01:35.187518 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.186805 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4fmfc\"" Jul 23 15:01:35.187518 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.186885 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 15:01:35.187518 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.186958 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 15:01:35.195522 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.195468 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vgt49"] Jul 23 15:01:35.257081 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.257045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46k7k\" (UniqueName: \"kubernetes.io/projected/3288c6c1-8f5a-4c86-98b2-96c54c49dda0-kube-api-access-46k7k\") pod \"model-serving-api-54d89bd5fc-c4p9v\" (UID: \"3288c6c1-8f5a-4c86-98b2-96c54c49dda0\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.257301 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.257153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288c6c1-8f5a-4c86-98b2-96c54c49dda0-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4p9v\" (UID: \"3288c6c1-8f5a-4c86-98b2-96c54c49dda0\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.260280 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.260239 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288c6c1-8f5a-4c86-98b2-96c54c49dda0-tls-certs\") pod \"model-serving-api-54d89bd5fc-c4p9v\" (UID: \"3288c6c1-8f5a-4c86-98b2-96c54c49dda0\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.269299 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.269208 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46k7k\" (UniqueName: \"kubernetes.io/projected/3288c6c1-8f5a-4c86-98b2-96c54c49dda0-kube-api-access-46k7k\") pod \"model-serving-api-54d89bd5fc-c4p9v\" (UID: \"3288c6c1-8f5a-4c86-98b2-96c54c49dda0\") " pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.284025 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.283949 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.357820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-token\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.357882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.357934 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.357972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-local-certs\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.357994 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-cacerts\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.358041 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl27m\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-kube-api-access-pl27m\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.358760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.358088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.434542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.434466 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" event={"ID":"4ce15863-c157-45e4-bc18-c2b2b92df82b","Type":"ContainerStarted","Data":"8b6d89cbc57b42d6d6e465bfb38bdc674b28a4cef5449f6806071c470328e75e"} Jul 23 15:01:35.450604 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.450543 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c4p9v"] Jul 23 15:01:35.456162 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:01:35.456131 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3288c6c1_8f5a_4c86_98b2_96c54c49dda0.slice/crio-64288127cb1d0efdbe8e6c0538bbd3086e4c1d6e7298c2dd9d8ec2c986a4f752 WatchSource:0}: Error finding container 64288127cb1d0efdbe8e6c0538bbd3086e4c1d6e7298c2dd9d8ec2c986a4f752: Status 404 returned error can't find the container with id 64288127cb1d0efdbe8e6c0538bbd3086e4c1d6e7298c2dd9d8ec2c986a4f752 Jul 23 15:01:35.458585 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl27m\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-kube-api-access-pl27m\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.458679 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.458679 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458676 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-token\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.458818 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458706 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.458818 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.458818 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458797 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-local-certs\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.458951 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.458820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-cacerts\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.459546 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.459520 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.461133 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.461110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-local-certs\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.461469 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.461447 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.461681 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.461655 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-cacerts\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.461734 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.461706 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.469006 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.468983 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-token\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.469345 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.469326 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl27m\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-kube-api-access-pl27m\") pod \"istiod-openshift-gateway-7574459656-vgt49\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.498111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.498073 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:35.673946 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:35.673916 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vgt49"] Jul 23 15:01:35.675967 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:01:35.675933 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b3ce32_5cc1_4550_aab0_a555bc17e2ae.slice/crio-86ce2c6bd44d49b2abe8de5734509ba600c7f4c7694fc782cba00acd2814cfd6 WatchSource:0}: Error finding container 86ce2c6bd44d49b2abe8de5734509ba600c7f4c7694fc782cba00acd2814cfd6: Status 404 returned error can't find the container with id 86ce2c6bd44d49b2abe8de5734509ba600c7f4c7694fc782cba00acd2814cfd6 Jul 23 15:01:36.441989 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:36.441944 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" event={"ID":"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae","Type":"ContainerStarted","Data":"86ce2c6bd44d49b2abe8de5734509ba600c7f4c7694fc782cba00acd2814cfd6"} Jul 23 15:01:36.443426 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:36.443390 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" event={"ID":"3288c6c1-8f5a-4c86-98b2-96c54c49dda0","Type":"ContainerStarted","Data":"64288127cb1d0efdbe8e6c0538bbd3086e4c1d6e7298c2dd9d8ec2c986a4f752"} Jul 23 15:01:37.450901 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:37.450867 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" event={"ID":"5dcc70a2-4867-4e0d-99fb-0005bd689a71","Type":"ContainerStarted","Data":"f951ca096e05694b87a7a194d4fc09e365ba47b4b6b29769d11e2531f1948397"} Jul 23 15:01:37.481414 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:37.481356 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8d89f64f-zpd4n" podStartSLOduration=1.92780775 podStartE2EDuration="4.481335694s" totalImagesPullingTime="2.553527944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:37.479797226 +0000 UTC m=+545.631246596" watchObservedRunningTime="2026-07-23 15:01:37.481335694 +0000 UTC m=+545.632785064" Jul 23 15:01:39.156646 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.156614 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 15:01:39.156930 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.156682 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 15:01:39.467559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.467424 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" event={"ID":"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae","Type":"ContainerStarted","Data":"eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c"} Jul 23 15:01:39.467559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.467511 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:39.469920 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.469820 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" event={"ID":"3288c6c1-8f5a-4c86-98b2-96c54c49dda0","Type":"ContainerStarted","Data":"e8db720647ac53e8a1bcdb2311021c167a029207892e70261f86fc9feac7fbb0"} Jul 23 15:01:39.470226 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.470167 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:39.501078 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.501016 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" podStartSLOduration=1.023070538 podStartE2EDuration="4.500994206s" totalImagesPullingTime="3.477923668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:39.49773991 +0000 UTC m=+547.649189280" watchObservedRunningTime="2026-07-23 15:01:39.500994206 +0000 UTC m=+547.652443578" Jul 23 15:01:39.517656 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:39.516908 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" podStartSLOduration=1.82352758 podStartE2EDuration="5.516883445s" totalImagesPullingTime="3.693355865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:39.515374776 +0000 UTC m=+547.666824184" watchObservedRunningTime="2026-07-23 15:01:39.516883445 +0000 UTC m=+547.668332814" Jul 23 15:01:40.484209 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:40.484087 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vgt49 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 15:01:40.484209 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:40.484142 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" podUID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:01:42.491061 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:42.491019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" event={"ID":"4ce15863-c157-45e4-bc18-c2b2b92df82b","Type":"ContainerStarted","Data":"fe0c950f24266b0abb93067910dd3e2c4793cdf66b4b7e51a0f178703a286ac1"} Jul 23 15:01:42.491540 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:42.491109 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:01:42.539685 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:42.539628 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" podStartSLOduration=3.140529236 podStartE2EDuration="9.5396113s" totalImagesPullingTime="6.399082064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:42.526053207 +0000 UTC m=+550.677502577" watchObservedRunningTime="2026-07-23 15:01:42.5396113 +0000 UTC m=+550.691060668" Jul 23 15:01:43.483638 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:43.483611 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:45.841329 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.841295 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc"] Jul 23 15:01:45.847314 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.847292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:45.851448 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.851416 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ql288\"" Jul 23 15:01:45.852866 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.852846 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 15:01:45.869217 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.869191 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc"] Jul 23 15:01:45.877996 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.877961 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2"] Jul 23 15:01:45.883980 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.883960 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:45.904530 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.904489 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 23 15:01:45.905109 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.905078 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d785f\"" Jul 23 15:01:45.912309 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.912036 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2"] Jul 23 15:01:45.961644 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.961608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d8257fa-d66f-4023-935a-f65d89354b55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vb9b2\" (UID: \"0d8257fa-d66f-4023-935a-f65d89354b55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:45.961845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.961661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc6kc\" (UniqueName: \"kubernetes.io/projected/0d8257fa-d66f-4023-935a-f65d89354b55-kube-api-access-fc6kc\") pod \"llmisvc-controller-manager-777c6bd7d5-vb9b2\" (UID: \"0d8257fa-d66f-4023-935a-f65d89354b55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:45.961845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.961699 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l299\" (UniqueName: \"kubernetes.io/projected/97bad5ce-6a36-484e-b9db-d017e9b81a8a-kube-api-access-2l299\") pod \"kserve-controller-manager-d6f4b65b8-vbvdc\" (UID: \"97bad5ce-6a36-484e-b9db-d017e9b81a8a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:45.961845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:45.961774 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97bad5ce-6a36-484e-b9db-d017e9b81a8a-cert\") pod \"kserve-controller-manager-d6f4b65b8-vbvdc\" (UID: \"97bad5ce-6a36-484e-b9db-d017e9b81a8a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:46.062999 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.062966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d8257fa-d66f-4023-935a-f65d89354b55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vb9b2\" (UID: \"0d8257fa-d66f-4023-935a-f65d89354b55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:46.063179 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.063009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc6kc\" (UniqueName: \"kubernetes.io/projected/0d8257fa-d66f-4023-935a-f65d89354b55-kube-api-access-fc6kc\") pod \"llmisvc-controller-manager-777c6bd7d5-vb9b2\" (UID: \"0d8257fa-d66f-4023-935a-f65d89354b55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:46.063179 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.063049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l299\" (UniqueName: \"kubernetes.io/projected/97bad5ce-6a36-484e-b9db-d017e9b81a8a-kube-api-access-2l299\") pod \"kserve-controller-manager-d6f4b65b8-vbvdc\" (UID: \"97bad5ce-6a36-484e-b9db-d017e9b81a8a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:46.063179 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.063083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97bad5ce-6a36-484e-b9db-d017e9b81a8a-cert\") pod \"kserve-controller-manager-d6f4b65b8-vbvdc\" (UID: \"97bad5ce-6a36-484e-b9db-d017e9b81a8a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:46.065747 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.065715 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97bad5ce-6a36-484e-b9db-d017e9b81a8a-cert\") pod \"kserve-controller-manager-d6f4b65b8-vbvdc\" (UID: \"97bad5ce-6a36-484e-b9db-d017e9b81a8a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:46.065845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.065786 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d8257fa-d66f-4023-935a-f65d89354b55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vb9b2\" (UID: \"0d8257fa-d66f-4023-935a-f65d89354b55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:46.086993 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.086960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l299\" (UniqueName: \"kubernetes.io/projected/97bad5ce-6a36-484e-b9db-d017e9b81a8a-kube-api-access-2l299\") pod \"kserve-controller-manager-d6f4b65b8-vbvdc\" (UID: \"97bad5ce-6a36-484e-b9db-d017e9b81a8a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:46.090205 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.090180 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc6kc\" (UniqueName: \"kubernetes.io/projected/0d8257fa-d66f-4023-935a-f65d89354b55-kube-api-access-fc6kc\") pod \"llmisvc-controller-manager-777c6bd7d5-vb9b2\" (UID: \"0d8257fa-d66f-4023-935a-f65d89354b55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:46.160648 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.160614 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:46.211308 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.211272 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:46.354837 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.354808 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc"] Jul 23 15:01:46.357048 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:01:46.357011 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bad5ce_6a36_484e_b9db_d017e9b81a8a.slice/crio-08c02ce2cc350e5678f78a075bf8ac38a32eae30f3ed5c6922b209f1112566cc WatchSource:0}: Error finding container 08c02ce2cc350e5678f78a075bf8ac38a32eae30f3ed5c6922b209f1112566cc: Status 404 returned error can't find the container with id 08c02ce2cc350e5678f78a075bf8ac38a32eae30f3ed5c6922b209f1112566cc Jul 23 15:01:46.390792 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.390766 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2"] Jul 23 15:01:46.392609 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:01:46.392575 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0d8257fa_d66f_4023_935a_f65d89354b55.slice/crio-03f2f965786dbb6c3bd966d338768b9e4d928563559c08702cbe818dea584880 WatchSource:0}: Error finding container 03f2f965786dbb6c3bd966d338768b9e4d928563559c08702cbe818dea584880: Status 404 returned error can't find the container with id 03f2f965786dbb6c3bd966d338768b9e4d928563559c08702cbe818dea584880 Jul 23 15:01:46.513078 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.512987 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" event={"ID":"97bad5ce-6a36-484e-b9db-d017e9b81a8a","Type":"ContainerStarted","Data":"08c02ce2cc350e5678f78a075bf8ac38a32eae30f3ed5c6922b209f1112566cc"} Jul 23 15:01:46.514185 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:46.514159 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" event={"ID":"0d8257fa-d66f-4023-935a-f65d89354b55","Type":"ContainerStarted","Data":"03f2f965786dbb6c3bd966d338768b9e4d928563559c08702cbe818dea584880"} Jul 23 15:01:47.417588 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.417538 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vgt49"] Jul 23 15:01:47.418055 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.417762 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" podUID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" containerName="discovery" containerID="cri-o://eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c" gracePeriod=30 Jul 23 15:01:47.693222 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.693195 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:47.780538 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780508 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-token\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.780703 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780574 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl27m\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-kube-api-access-pl27m\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.780703 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780596 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-local-certs\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.780703 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780643 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-dns-cert\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.780850 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780709 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-cacerts\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.780850 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780745 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-ca-configmap\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.780850 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.780773 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-kubeconfig\") pod \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\" (UID: \"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae\") " Jul 23 15:01:47.781308 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.781231 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:01:47.783331 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.783295 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:01:47.783449 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.783387 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:01:47.783591 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.783570 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-kube-api-access-pl27m" (OuterVolumeSpecName: "kube-api-access-pl27m") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "kube-api-access-pl27m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:01:47.783651 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.783573 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-cacerts" (OuterVolumeSpecName: "cacerts") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:01:47.783703 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.783691 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:01:47.783759 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.783740 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-local-certs" (OuterVolumeSpecName: "local-certs") pod "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" (UID: "f9b3ce32-5cc1-4550-aab0-a555bc17e2ae"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884387 2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-kubeconfig\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884424 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-token\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884439 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl27m\" (UniqueName: \"kubernetes.io/projected/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-kube-api-access-pl27m\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884464 2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-local-certs\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884477 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-dns-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884489 2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-cacerts\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:47.884528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:47.884502 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae-istio-csr-ca-configmap\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:01:48.528333 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.528281 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" containerID="eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c" exitCode=0 Jul 23 15:01:48.528786 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.528339 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" Jul 23 15:01:48.528786 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.528367 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" event={"ID":"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae","Type":"ContainerDied","Data":"eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c"} Jul 23 15:01:48.528786 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.528438 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vgt49" event={"ID":"f9b3ce32-5cc1-4550-aab0-a555bc17e2ae","Type":"ContainerDied","Data":"86ce2c6bd44d49b2abe8de5734509ba600c7f4c7694fc782cba00acd2814cfd6"} Jul 23 15:01:48.528786 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.528466 2582 scope.go:117] "RemoveContainer" containerID="eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c" Jul 23 15:01:48.553635 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.553553 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vgt49"] Jul 23 15:01:48.559866 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.559838 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vgt49"] Jul 23 15:01:48.797682 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.797654 2582 scope.go:117] "RemoveContainer" containerID="eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c" Jul 23 15:01:48.798030 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:01:48.798013 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c\": container with ID starting with eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c not found: ID does not exist" containerID="eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c" Jul 23 15:01:48.798076 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:48.798040 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c"} err="failed to get container status \"eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c\": rpc error: code = NotFound desc = could not find container \"eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c\": container with ID starting with eb681415a800f24ef4d509bd2af8bd6df347ee798385f06ed4d06d1381cf3c0c not found: ID does not exist" Jul 23 15:01:49.535621 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:49.535583 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" event={"ID":"97bad5ce-6a36-484e-b9db-d017e9b81a8a","Type":"ContainerStarted","Data":"0d08c706dd5f91f0d1031bcdbbf82e56d240dd7cbea3789619526e969eb16a03"} Jul 23 15:01:49.536155 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:49.535649 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:01:49.555807 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:49.555759 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" podStartSLOduration=2.069102656 podStartE2EDuration="4.555745069s" totalImagesPullingTime="2.486642413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:49.554496081 +0000 UTC m=+557.705945453" watchObservedRunningTime="2026-07-23 15:01:49.555745069 +0000 UTC m=+557.707194438" Jul 23 15:01:50.295177 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:50.295142 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" path="/var/lib/kubelet/pods/f9b3ce32-5cc1-4550-aab0-a555bc17e2ae/volumes" Jul 23 15:01:50.487023 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:50.486963 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c4p9v" Jul 23 15:01:51.548894 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:51.548862 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" event={"ID":"0d8257fa-d66f-4023-935a-f65d89354b55","Type":"ContainerStarted","Data":"e8f78f5983fb9b26949a5b3de554b8c2812484b86543eb4562ed0f307d031aa3"} Jul 23 15:01:51.549313 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:51.548946 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:01:51.567576 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:51.567524 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" podStartSLOduration=1.920042369 podStartE2EDuration="6.567506701s" totalImagesPullingTime="4.647464332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:01:51.565531399 +0000 UTC m=+559.716980768" watchObservedRunningTime="2026-07-23 15:01:51.567506701 +0000 UTC m=+559.718956071" Jul 23 15:01:53.499190 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:01:53.499156 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-vgvfp" Jul 23 15:02:20.547748 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:20.547717 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vbvdc" Jul 23 15:02:22.556409 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:22.556375 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vb9b2" Jul 23 15:02:32.258583 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:32.258558 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:02:32.259781 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:32.259764 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:02:34.983177 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.983143 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtwtn"] Jul 23 15:02:34.984867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.983671 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" containerName="discovery" Jul 23 15:02:34.984867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.983686 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" containerName="discovery" Jul 23 15:02:34.984867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.983744 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b3ce32-5cc1-4550-aab0-a555bc17e2ae" containerName="discovery" Jul 23 15:02:34.985800 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.985785 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:34.988906 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.988882 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 15:02:34.989949 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.989926 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hvk2h\"" Jul 23 15:02:34.990074 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.989926 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 15:02:34.995558 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:34.995524 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtwtn"] Jul 23 15:02:35.091936 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.091900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgfk9\" (UniqueName: \"kubernetes.io/projected/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6-kube-api-access-rgfk9\") pod \"kuadrant-operator-catalog-vtwtn\" (UID: \"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6\") " pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:35.192581 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.192543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgfk9\" (UniqueName: \"kubernetes.io/projected/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6-kube-api-access-rgfk9\") pod \"kuadrant-operator-catalog-vtwtn\" (UID: \"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6\") " pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:35.202018 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.201990 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgfk9\" (UniqueName: \"kubernetes.io/projected/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6-kube-api-access-rgfk9\") pod \"kuadrant-operator-catalog-vtwtn\" (UID: \"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6\") " pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:35.297199 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.297113 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:35.345042 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.345003 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtwtn"] Jul 23 15:02:35.430616 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.430591 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtwtn"] Jul 23 15:02:35.432969 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:02:35.432941 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbd0af5_7c2e_4137_a329_a2e67b5ffba6.slice/crio-db1dcdb3b70f85ad7a7c9c8e3e0ea5aa8279e64503e1e2cdb6a6185c217772d8 WatchSource:0}: Error finding container db1dcdb3b70f85ad7a7c9c8e3e0ea5aa8279e64503e1e2cdb6a6185c217772d8: Status 404 returned error can't find the container with id db1dcdb3b70f85ad7a7c9c8e3e0ea5aa8279e64503e1e2cdb6a6185c217772d8 Jul 23 15:02:35.434867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.434844 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:02:35.557308 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.557212 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6rxsf"] Jul 23 15:02:35.561010 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.560996 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:35.567701 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.567680 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6rxsf"] Jul 23 15:02:35.595195 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.595164 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5ll4\" (UniqueName: \"kubernetes.io/projected/e9f59e05-5769-48ae-9789-2e71a4a72928-kube-api-access-w5ll4\") pod \"kuadrant-operator-catalog-6rxsf\" (UID: \"e9f59e05-5769-48ae-9789-2e71a4a72928\") " pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:35.696470 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.696440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5ll4\" (UniqueName: \"kubernetes.io/projected/e9f59e05-5769-48ae-9789-2e71a4a72928-kube-api-access-w5ll4\") pod \"kuadrant-operator-catalog-6rxsf\" (UID: \"e9f59e05-5769-48ae-9789-2e71a4a72928\") " pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:35.706532 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.706510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5ll4\" (UniqueName: \"kubernetes.io/projected/e9f59e05-5769-48ae-9789-2e71a4a72928-kube-api-access-w5ll4\") pod \"kuadrant-operator-catalog-6rxsf\" (UID: \"e9f59e05-5769-48ae-9789-2e71a4a72928\") " pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:35.764686 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.764645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" event={"ID":"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6","Type":"ContainerStarted","Data":"db1dcdb3b70f85ad7a7c9c8e3e0ea5aa8279e64503e1e2cdb6a6185c217772d8"} Jul 23 15:02:35.872073 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:35.872008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:36.161117 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:36.161090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6rxsf"] Jul 23 15:02:36.201357 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:02:36.201323 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f59e05_5769_48ae_9789_2e71a4a72928.slice/crio-0fc3ce48365e6ed0133688e4abd336241229f228792c75424a54be2b9ba48616 WatchSource:0}: Error finding container 0fc3ce48365e6ed0133688e4abd336241229f228792c75424a54be2b9ba48616: Status 404 returned error can't find the container with id 0fc3ce48365e6ed0133688e4abd336241229f228792c75424a54be2b9ba48616 Jul 23 15:02:36.772732 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:36.772559 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" event={"ID":"e9f59e05-5769-48ae-9789-2e71a4a72928","Type":"ContainerStarted","Data":"0fc3ce48365e6ed0133688e4abd336241229f228792c75424a54be2b9ba48616"} Jul 23 15:02:37.784736 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:37.784698 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" event={"ID":"e9f59e05-5769-48ae-9789-2e71a4a72928","Type":"ContainerStarted","Data":"611a42c269419711b35c2ac8f2439870ee15b98855d1d3b97be182a69c5192ba"} Jul 23 15:02:37.803293 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:37.803223 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" podStartSLOduration=1.363200816 podStartE2EDuration="2.803206336s" totalImagesPullingTime="1.44000552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:02:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:02:37.800590606 +0000 UTC m=+605.952039974" watchObservedRunningTime="2026-07-23 15:02:37.803206336 +0000 UTC m=+605.954655728" Jul 23 15:02:39.795927 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:39.795870 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" event={"ID":"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6","Type":"ContainerStarted","Data":"4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663"} Jul 23 15:02:39.795927 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:39.795907 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" podUID="fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" containerName="registry-server" containerID="cri-o://4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663" gracePeriod=2 Jul 23 15:02:39.815202 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:39.815150 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" podStartSLOduration=2.180907944 podStartE2EDuration="5.815132327s" totalImagesPullingTime="3.634224383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:02:39.814228241 +0000 UTC m=+607.965677611" watchObservedRunningTime="2026-07-23 15:02:39.815132327 +0000 UTC m=+607.966581695" Jul 23 15:02:40.050961 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.050898 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:40.134575 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.134535 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgfk9\" (UniqueName: \"kubernetes.io/projected/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6-kube-api-access-rgfk9\") pod \"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6\" (UID: \"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6\") " Jul 23 15:02:40.136874 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.136842 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6-kube-api-access-rgfk9" (OuterVolumeSpecName: "kube-api-access-rgfk9") pod "fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" (UID: "fbbd0af5-7c2e-4137-a329-a2e67b5ffba6"). InnerVolumeSpecName "kube-api-access-rgfk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:02:40.235129 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.235101 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgfk9\" (UniqueName: \"kubernetes.io/projected/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6-kube-api-access-rgfk9\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:40.801934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.801894 2582 generic.go:356] "Generic (PLEG): container finished" podID="fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" containerID="4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663" exitCode=0 Jul 23 15:02:40.802385 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.801940 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" Jul 23 15:02:40.802385 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.801981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" event={"ID":"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6","Type":"ContainerDied","Data":"4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663"} Jul 23 15:02:40.802385 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.802038 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtwtn" event={"ID":"fbbd0af5-7c2e-4137-a329-a2e67b5ffba6","Type":"ContainerDied","Data":"db1dcdb3b70f85ad7a7c9c8e3e0ea5aa8279e64503e1e2cdb6a6185c217772d8"} Jul 23 15:02:40.802385 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.802059 2582 scope.go:117] "RemoveContainer" containerID="4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663" Jul 23 15:02:40.811401 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.811382 2582 scope.go:117] "RemoveContainer" containerID="4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663" Jul 23 15:02:40.811710 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:02:40.811676 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663\": container with ID starting with 4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663 not found: ID does not exist" containerID="4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663" Jul 23 15:02:40.811771 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.811718 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663"} err="failed to get container status \"4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663\": rpc error: code = NotFound desc = could not find container \"4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663\": container with ID starting with 4b7bb5b5d38b8cbfdd2f5a9054109c815199e025bdb92adb67e44592f9677663 not found: ID does not exist" Jul 23 15:02:40.821449 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.821421 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtwtn"] Jul 23 15:02:40.826208 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:40.826185 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtwtn"] Jul 23 15:02:42.293983 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:42.293950 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" path="/var/lib/kubelet/pods/fbbd0af5-7c2e-4137-a329-a2e67b5ffba6/volumes" Jul 23 15:02:45.872166 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:45.872129 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:45.872594 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:45.872188 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:45.895120 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:45.895096 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:46.854416 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:46.854390 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6rxsf" Jul 23 15:02:50.605553 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.605522 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk"] Jul 23 15:02:50.605936 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.605904 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" containerName="registry-server" Jul 23 15:02:50.605936 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.605916 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" containerName="registry-server" Jul 23 15:02:50.606002 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.605990 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbbd0af5-7c2e-4137-a329-a2e67b5ffba6" containerName="registry-server" Jul 23 15:02:50.610842 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.610818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.613706 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.613686 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f9sh9\"" Jul 23 15:02:50.617883 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.617860 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk"] Jul 23 15:02:50.730458 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.730419 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q7wn\" (UniqueName: \"kubernetes.io/projected/5b95b16b-0ff8-4133-a082-823ab45a9bb4-kube-api-access-5q7wn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.730630 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.730469 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.730630 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.730506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.831170 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.831133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q7wn\" (UniqueName: \"kubernetes.io/projected/5b95b16b-0ff8-4133-a082-823ab45a9bb4-kube-api-access-5q7wn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.831366 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.831184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.831366 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.831202 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.831556 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.831537 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.831607 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.831571 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.840587 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.840550 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q7wn\" (UniqueName: \"kubernetes.io/projected/5b95b16b-0ff8-4133-a082-823ab45a9bb4-kube-api-access-5q7wn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.921824 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.921793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:50.998615 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:50.998586 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6"] Jul 23 15:02:51.007272 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.007220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.010193 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.010165 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6"] Jul 23 15:02:51.066152 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.066129 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk"] Jul 23 15:02:51.067983 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:02:51.067956 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b95b16b_0ff8_4133_a082_823ab45a9bb4.slice/crio-60fd600724de25f887668804bb6bc839dad4dff3f8416099beda1c4d0214ece1 WatchSource:0}: Error finding container 60fd600724de25f887668804bb6bc839dad4dff3f8416099beda1c4d0214ece1: Status 404 returned error can't find the container with id 60fd600724de25f887668804bb6bc839dad4dff3f8416099beda1c4d0214ece1 Jul 23 15:02:51.135422 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.135398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.135536 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.135482 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5jd2\" (UniqueName: \"kubernetes.io/projected/800330fc-eb5c-455a-a0da-b2784726bf08-kube-api-access-z5jd2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.135536 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.135521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.236025 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.235939 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.236025 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.236005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5jd2\" (UniqueName: \"kubernetes.io/projected/800330fc-eb5c-455a-a0da-b2784726bf08-kube-api-access-z5jd2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.236222 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.236029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.236378 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.236349 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.236492 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.236416 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.246684 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.246663 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5jd2\" (UniqueName: \"kubernetes.io/projected/800330fc-eb5c-455a-a0da-b2784726bf08-kube-api-access-z5jd2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.320301 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.320273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:02:51.398395 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.398361 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8"] Jul 23 15:02:51.403035 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.403010 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.410421 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.410394 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8"] Jul 23 15:02:51.463737 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.463594 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6"] Jul 23 15:02:51.466359 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:02:51.466333 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800330fc_eb5c_455a_a0da_b2784726bf08.slice/crio-4d8e35cf6354f2cccf9f768c1397c96945005df647d5bb55bfe2a7615eb38dce WatchSource:0}: Error finding container 4d8e35cf6354f2cccf9f768c1397c96945005df647d5bb55bfe2a7615eb38dce: Status 404 returned error can't find the container with id 4d8e35cf6354f2cccf9f768c1397c96945005df647d5bb55bfe2a7615eb38dce Jul 23 15:02:51.538542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.538515 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txwcn\" (UniqueName: \"kubernetes.io/projected/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-kube-api-access-txwcn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.538682 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.538570 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.538731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.538678 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.639121 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.639086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.639542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.639167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.639542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.639219 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txwcn\" (UniqueName: \"kubernetes.io/projected/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-kube-api-access-txwcn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.639542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.639527 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.639636 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.639531 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.648881 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.648862 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txwcn\" (UniqueName: \"kubernetes.io/projected/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-kube-api-access-txwcn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.715555 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.715524 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:51.799616 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.799586 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7"] Jul 23 15:02:51.804432 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.804403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:51.814009 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.813947 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7"] Jul 23 15:02:51.858035 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.858008 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8"] Jul 23 15:02:51.859427 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.859395 2582 generic.go:356] "Generic (PLEG): container finished" podID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerID="9d94bcb2910a9924354858211a1ef309448d41ea0c91bbef83e0d072e5871db6" exitCode=0 Jul 23 15:02:51.859538 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.859455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" event={"ID":"5b95b16b-0ff8-4133-a082-823ab45a9bb4","Type":"ContainerDied","Data":"9d94bcb2910a9924354858211a1ef309448d41ea0c91bbef83e0d072e5871db6"} Jul 23 15:02:51.859538 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.859509 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" event={"ID":"5b95b16b-0ff8-4133-a082-823ab45a9bb4","Type":"ContainerStarted","Data":"60fd600724de25f887668804bb6bc839dad4dff3f8416099beda1c4d0214ece1"} Jul 23 15:02:51.860065 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:02:51.860023 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe2ddb3_18ac_4fc8_8ee3_2e2bb85f856a.slice/crio-43e2b00c746ea1ae123043415a4cbbaec84804ad9e0d2662ce0d6bfbfb2acf4f WatchSource:0}: Error finding container 43e2b00c746ea1ae123043415a4cbbaec84804ad9e0d2662ce0d6bfbfb2acf4f: Status 404 returned error can't find the container with id 43e2b00c746ea1ae123043415a4cbbaec84804ad9e0d2662ce0d6bfbfb2acf4f Jul 23 15:02:51.861247 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.861222 2582 generic.go:356] "Generic (PLEG): container finished" podID="800330fc-eb5c-455a-a0da-b2784726bf08" containerID="e679be5adeac014076218d90ad8b569463db4c03e6c0f52518386eb677d58249" exitCode=0 Jul 23 15:02:51.861371 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.861291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" event={"ID":"800330fc-eb5c-455a-a0da-b2784726bf08","Type":"ContainerDied","Data":"e679be5adeac014076218d90ad8b569463db4c03e6c0f52518386eb677d58249"} Jul 23 15:02:51.861371 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.861342 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" event={"ID":"800330fc-eb5c-455a-a0da-b2784726bf08","Type":"ContainerStarted","Data":"4d8e35cf6354f2cccf9f768c1397c96945005df647d5bb55bfe2a7615eb38dce"} Jul 23 15:02:51.941580 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.941544 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:51.941763 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.941661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:51.941763 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:51.941701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj29g\" (UniqueName: \"kubernetes.io/projected/ec980b45-51fc-4865-9eee-f00fec4061aa-kube-api-access-fj29g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.042510 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.042423 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.042510 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.042494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.042680 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.042531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj29g\" (UniqueName: \"kubernetes.io/projected/ec980b45-51fc-4865-9eee-f00fec4061aa-kube-api-access-fj29g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.042892 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.042871 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.042927 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.042889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.052021 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.051999 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj29g\" (UniqueName: \"kubernetes.io/projected/ec980b45-51fc-4865-9eee-f00fec4061aa-kube-api-access-fj29g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.116637 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.116604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:52.252272 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.252232 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7"] Jul 23 15:02:52.253508 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:02:52.253481 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec980b45_51fc_4865_9eee_f00fec4061aa.slice/crio-ef94cc51938d83277ff1e656055272d72065c5efeba36d705968aa57ae336c00 WatchSource:0}: Error finding container ef94cc51938d83277ff1e656055272d72065c5efeba36d705968aa57ae336c00: Status 404 returned error can't find the container with id ef94cc51938d83277ff1e656055272d72065c5efeba36d705968aa57ae336c00 Jul 23 15:02:52.678174 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:02:52.678132 2582 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/kuadrant/limitador-operator-bundle:v0.17.1: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" image="quay.io/kuadrant/limitador-operator-bundle:v0.17.1" Jul 23 15:02:52.678627 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:02:52.678243 2582 kuberuntime_manager.go:1459] "Unhandled Error" err="init container pull start failed in pod 0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_kuadrant-system(800330fc-eb5c-455a-a0da-b2784726bf08): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/kuadrant/limitador-operator-bundle:v0.17.1: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 23 15:02:52.678627 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:02:52.678306 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"pull\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/kuadrant/limitador-operator-bundle:v0.17.1: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" Jul 23 15:02:52.868760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.868729 2582 generic.go:356] "Generic (PLEG): container finished" podID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerID="b0b5e53bf86a3f1112de8a6a853db74921fba0f7b58bfa9d47be64ee73fc2113" exitCode=0 Jul 23 15:02:52.868894 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.868819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" event={"ID":"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a","Type":"ContainerDied","Data":"b0b5e53bf86a3f1112de8a6a853db74921fba0f7b58bfa9d47be64ee73fc2113"} Jul 23 15:02:52.868943 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.868903 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" event={"ID":"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a","Type":"ContainerStarted","Data":"43e2b00c746ea1ae123043415a4cbbaec84804ad9e0d2662ce0d6bfbfb2acf4f"} Jul 23 15:02:52.870458 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.870435 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerID="df6b00b5c8a248cfa4a8423ca8a6d6c7f9e58e17da372c51d1d7ed9253a6aa5e" exitCode=0 Jul 23 15:02:52.870572 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.870521 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" event={"ID":"ec980b45-51fc-4865-9eee-f00fec4061aa","Type":"ContainerDied","Data":"df6b00b5c8a248cfa4a8423ca8a6d6c7f9e58e17da372c51d1d7ed9253a6aa5e"} Jul 23 15:02:52.870572 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:52.870568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" event={"ID":"ec980b45-51fc-4865-9eee-f00fec4061aa","Type":"ContainerStarted","Data":"ef94cc51938d83277ff1e656055272d72065c5efeba36d705968aa57ae336c00"} Jul 23 15:02:52.871119 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:02:52.871092 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"pull\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/kuadrant/limitador-operator-bundle:v0.17.1\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/kuadrant/limitador-operator-bundle:v0.17.1: Requesting bearer token: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" Jul 23 15:02:53.882560 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:53.882532 2582 generic.go:356] "Generic (PLEG): container finished" podID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerID="9eabeff67851605d4c2ac2c42e4e7bf3283480ade4e67d31373acb68d065940d" exitCode=0 Jul 23 15:02:53.882941 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:53.882614 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" event={"ID":"5b95b16b-0ff8-4133-a082-823ab45a9bb4","Type":"ContainerDied","Data":"9eabeff67851605d4c2ac2c42e4e7bf3283480ade4e67d31373acb68d065940d"} Jul 23 15:02:54.889522 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:54.889491 2582 generic.go:356] "Generic (PLEG): container finished" podID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerID="dce47a246e4c8abf1a5e9705cf0844d96b50ae93055bb4edfd212dc1ae8ffa0f" exitCode=0 Jul 23 15:02:54.889955 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:54.889581 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" event={"ID":"5b95b16b-0ff8-4133-a082-823ab45a9bb4","Type":"ContainerDied","Data":"dce47a246e4c8abf1a5e9705cf0844d96b50ae93055bb4edfd212dc1ae8ffa0f"} Jul 23 15:02:54.891204 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:54.891180 2582 generic.go:356] "Generic (PLEG): container finished" podID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerID="91a3e9a1f5e4941390bca9ac5e37473ebb3a24aa1548e338db235b596db0b690" exitCode=0 Jul 23 15:02:54.891338 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:54.891279 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" event={"ID":"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a","Type":"ContainerDied","Data":"91a3e9a1f5e4941390bca9ac5e37473ebb3a24aa1548e338db235b596db0b690"} Jul 23 15:02:54.893077 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:54.893053 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerID="1c02d264da77016230395df7a3c5c96c697d63e4fe0272116073647fd46c93bc" exitCode=0 Jul 23 15:02:54.893193 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:54.893098 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" event={"ID":"ec980b45-51fc-4865-9eee-f00fec4061aa","Type":"ContainerDied","Data":"1c02d264da77016230395df7a3c5c96c697d63e4fe0272116073647fd46c93bc"} Jul 23 15:02:55.899921 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:55.899890 2582 generic.go:356] "Generic (PLEG): container finished" podID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerID="b9da04006a3fc24747a7163b73cc6cc1115fa35287c22179a9004229fc07adb0" exitCode=0 Jul 23 15:02:55.900296 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:55.899972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" event={"ID":"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a","Type":"ContainerDied","Data":"b9da04006a3fc24747a7163b73cc6cc1115fa35287c22179a9004229fc07adb0"} Jul 23 15:02:55.902009 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:55.901988 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerID="6d48541b97a7d25938e91eee6296b684f3c238a8bdc2196233c34a0023e7f087" exitCode=0 Jul 23 15:02:55.902117 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:55.902057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" event={"ID":"ec980b45-51fc-4865-9eee-f00fec4061aa","Type":"ContainerDied","Data":"6d48541b97a7d25938e91eee6296b684f3c238a8bdc2196233c34a0023e7f087"} Jul 23 15:02:56.034646 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.034619 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:56.184845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.184743 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q7wn\" (UniqueName: \"kubernetes.io/projected/5b95b16b-0ff8-4133-a082-823ab45a9bb4-kube-api-access-5q7wn\") pod \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " Jul 23 15:02:56.184845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.184825 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-util\") pod \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " Jul 23 15:02:56.185053 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.184885 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-bundle\") pod \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\" (UID: \"5b95b16b-0ff8-4133-a082-823ab45a9bb4\") " Jul 23 15:02:56.185374 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.185346 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-bundle" (OuterVolumeSpecName: "bundle") pod "5b95b16b-0ff8-4133-a082-823ab45a9bb4" (UID: "5b95b16b-0ff8-4133-a082-823ab45a9bb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:02:56.187100 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.187076 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b95b16b-0ff8-4133-a082-823ab45a9bb4-kube-api-access-5q7wn" (OuterVolumeSpecName: "kube-api-access-5q7wn") pod "5b95b16b-0ff8-4133-a082-823ab45a9bb4" (UID: "5b95b16b-0ff8-4133-a082-823ab45a9bb4"). InnerVolumeSpecName "kube-api-access-5q7wn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:02:56.190312 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.190291 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-util" (OuterVolumeSpecName: "util") pod "5b95b16b-0ff8-4133-a082-823ab45a9bb4" (UID: "5b95b16b-0ff8-4133-a082-823ab45a9bb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:02:56.285453 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.285414 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:56.285453 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.285449 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q7wn\" (UniqueName: \"kubernetes.io/projected/5b95b16b-0ff8-4133-a082-823ab45a9bb4-kube-api-access-5q7wn\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:56.285453 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.285463 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b95b16b-0ff8-4133-a082-823ab45a9bb4-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:56.909470 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.909437 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" Jul 23 15:02:56.909470 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.909455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk" event={"ID":"5b95b16b-0ff8-4133-a082-823ab45a9bb4","Type":"ContainerDied","Data":"60fd600724de25f887668804bb6bc839dad4dff3f8416099beda1c4d0214ece1"} Jul 23 15:02:56.909972 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:56.909511 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60fd600724de25f887668804bb6bc839dad4dff3f8416099beda1c4d0214ece1" Jul 23 15:02:57.055884 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.055860 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:57.087193 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.087167 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:57.193576 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.193493 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-bundle\") pod \"ec980b45-51fc-4865-9eee-f00fec4061aa\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " Jul 23 15:02:57.193576 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.193534 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-util\") pod \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " Jul 23 15:02:57.193576 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.193560 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txwcn\" (UniqueName: \"kubernetes.io/projected/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-kube-api-access-txwcn\") pod \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " Jul 23 15:02:57.193838 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.193584 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-bundle\") pod \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\" (UID: \"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a\") " Jul 23 15:02:57.193838 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.193609 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-util\") pod \"ec980b45-51fc-4865-9eee-f00fec4061aa\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " Jul 23 15:02:57.193838 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.193688 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj29g\" (UniqueName: \"kubernetes.io/projected/ec980b45-51fc-4865-9eee-f00fec4061aa-kube-api-access-fj29g\") pod \"ec980b45-51fc-4865-9eee-f00fec4061aa\" (UID: \"ec980b45-51fc-4865-9eee-f00fec4061aa\") " Jul 23 15:02:57.194217 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.194186 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-bundle" (OuterVolumeSpecName: "bundle") pod "ec980b45-51fc-4865-9eee-f00fec4061aa" (UID: "ec980b45-51fc-4865-9eee-f00fec4061aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:02:57.194414 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.194309 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-bundle" (OuterVolumeSpecName: "bundle") pod "efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" (UID: "efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:02:57.195932 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.195901 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec980b45-51fc-4865-9eee-f00fec4061aa-kube-api-access-fj29g" (OuterVolumeSpecName: "kube-api-access-fj29g") pod "ec980b45-51fc-4865-9eee-f00fec4061aa" (UID: "ec980b45-51fc-4865-9eee-f00fec4061aa"). InnerVolumeSpecName "kube-api-access-fj29g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:02:57.196029 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.195934 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-kube-api-access-txwcn" (OuterVolumeSpecName: "kube-api-access-txwcn") pod "efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" (UID: "efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a"). InnerVolumeSpecName "kube-api-access-txwcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:02:57.201615 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.201585 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-util" (OuterVolumeSpecName: "util") pod "ec980b45-51fc-4865-9eee-f00fec4061aa" (UID: "ec980b45-51fc-4865-9eee-f00fec4061aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:02:57.295361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.295326 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:57.295361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.295355 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txwcn\" (UniqueName: \"kubernetes.io/projected/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-kube-api-access-txwcn\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:57.295361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.295365 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:57.295361 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.295374 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec980b45-51fc-4865-9eee-f00fec4061aa-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:57.295644 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.295384 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj29g\" (UniqueName: \"kubernetes.io/projected/ec980b45-51fc-4865-9eee-f00fec4061aa-kube-api-access-fj29g\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:57.304002 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.303973 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-util" (OuterVolumeSpecName: "util") pod "efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" (UID: "efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:02:57.396409 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.396373 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:02:57.916762 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.916729 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" Jul 23 15:02:57.917200 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.916737 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8" event={"ID":"efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a","Type":"ContainerDied","Data":"43e2b00c746ea1ae123043415a4cbbaec84804ad9e0d2662ce0d6bfbfb2acf4f"} Jul 23 15:02:57.917200 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.916868 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43e2b00c746ea1ae123043415a4cbbaec84804ad9e0d2662ce0d6bfbfb2acf4f" Jul 23 15:02:57.918810 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.918787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" event={"ID":"ec980b45-51fc-4865-9eee-f00fec4061aa","Type":"ContainerDied","Data":"ef94cc51938d83277ff1e656055272d72065c5efeba36d705968aa57ae336c00"} Jul 23 15:02:57.918912 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.918823 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7" Jul 23 15:02:57.918912 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:02:57.918829 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef94cc51938d83277ff1e656055272d72065c5efeba36d705968aa57ae336c00" Jul 23 15:03:08.984233 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:08.984145 2582 generic.go:356] "Generic (PLEG): container finished" podID="800330fc-eb5c-455a-a0da-b2784726bf08" containerID="0773e536b3b2c556b204f5743e48819aa99acc535915087917bad7cb48bc6066" exitCode=0 Jul 23 15:03:08.984233 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:08.984171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" event={"ID":"800330fc-eb5c-455a-a0da-b2784726bf08","Type":"ContainerDied","Data":"0773e536b3b2c556b204f5743e48819aa99acc535915087917bad7cb48bc6066"} Jul 23 15:03:09.991468 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:09.991436 2582 generic.go:356] "Generic (PLEG): container finished" podID="800330fc-eb5c-455a-a0da-b2784726bf08" containerID="14dd3162df2112fa007841de1e56f68c15355a7e12de5eb3f299d68bed16ccf8" exitCode=0 Jul 23 15:03:09.991866 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:09.991507 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" event={"ID":"800330fc-eb5c-455a-a0da-b2784726bf08","Type":"ContainerDied","Data":"14dd3162df2112fa007841de1e56f68c15355a7e12de5eb3f299d68bed16ccf8"} Jul 23 15:03:11.135788 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.135762 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:03:11.217974 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.217940 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5jd2\" (UniqueName: \"kubernetes.io/projected/800330fc-eb5c-455a-a0da-b2784726bf08-kube-api-access-z5jd2\") pod \"800330fc-eb5c-455a-a0da-b2784726bf08\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " Jul 23 15:03:11.217974 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.217978 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-bundle\") pod \"800330fc-eb5c-455a-a0da-b2784726bf08\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " Jul 23 15:03:11.218186 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.218030 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-util\") pod \"800330fc-eb5c-455a-a0da-b2784726bf08\" (UID: \"800330fc-eb5c-455a-a0da-b2784726bf08\") " Jul 23 15:03:11.218532 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.218508 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-bundle" (OuterVolumeSpecName: "bundle") pod "800330fc-eb5c-455a-a0da-b2784726bf08" (UID: "800330fc-eb5c-455a-a0da-b2784726bf08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:03:11.220269 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.220229 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/800330fc-eb5c-455a-a0da-b2784726bf08-kube-api-access-z5jd2" (OuterVolumeSpecName: "kube-api-access-z5jd2") pod "800330fc-eb5c-455a-a0da-b2784726bf08" (UID: "800330fc-eb5c-455a-a0da-b2784726bf08"). InnerVolumeSpecName "kube-api-access-z5jd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:03:11.222537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.222511 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-util" (OuterVolumeSpecName: "util") pod "800330fc-eb5c-455a-a0da-b2784726bf08" (UID: "800330fc-eb5c-455a-a0da-b2784726bf08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:03:11.318725 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.318647 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5jd2\" (UniqueName: \"kubernetes.io/projected/800330fc-eb5c-455a-a0da-b2784726bf08-kube-api-access-z5jd2\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:11.318725 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.318673 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-bundle\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:11.318725 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:11.318684 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/800330fc-eb5c-455a-a0da-b2784726bf08-util\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:12.004938 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:12.004899 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" event={"ID":"800330fc-eb5c-455a-a0da-b2784726bf08","Type":"ContainerDied","Data":"4d8e35cf6354f2cccf9f768c1397c96945005df647d5bb55bfe2a7615eb38dce"} Jul 23 15:03:12.005114 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:12.004962 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d8e35cf6354f2cccf9f768c1397c96945005df647d5bb55bfe2a7615eb38dce" Jul 23 15:03:12.005114 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:12.004919 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6" Jul 23 15:03:17.742861 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.742831 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw"] Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743164 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="extract" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743175 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="extract" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743183 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="pull" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743188 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="pull" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743199 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="util" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743204 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="util" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743217 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="util" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743222 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="util" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743230 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="extract" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743235 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="extract" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743243 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="util" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743249 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="util" Jul 23 15:03:17.743278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743283 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="pull" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743291 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="pull" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743302 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="pull" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743310 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="pull" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743322 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743327 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743336 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="util" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743341 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="util" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743346 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="pull" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743351 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="pull" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743356 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743360 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743436 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743442 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="800330fc-eb5c-455a-a0da-b2784726bf08" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743452 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b95b16b-0ff8-4133-a082-823ab45a9bb4" containerName="extract" Jul 23 15:03:17.743721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.743458 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec980b45-51fc-4865-9eee-f00fec4061aa" containerName="extract" Jul 23 15:03:17.745622 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.745604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:17.750834 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.750811 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p8fs6\"" Jul 23 15:03:17.758841 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.758819 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw"] Jul 23 15:03:17.775555 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.775526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9jx2\" (UniqueName: \"kubernetes.io/projected/a44926b7-ece0-4371-89be-a57a8f5cd019-kube-api-access-p9jx2\") pod \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" (UID: \"a44926b7-ece0-4371-89be-a57a8f5cd019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:17.876705 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.876669 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9jx2\" (UniqueName: \"kubernetes.io/projected/a44926b7-ece0-4371-89be-a57a8f5cd019-kube-api-access-p9jx2\") pod \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" (UID: \"a44926b7-ece0-4371-89be-a57a8f5cd019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:17.888770 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:17.888745 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9jx2\" (UniqueName: \"kubernetes.io/projected/a44926b7-ece0-4371-89be-a57a8f5cd019-kube-api-access-p9jx2\") pod \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" (UID: \"a44926b7-ece0-4371-89be-a57a8f5cd019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:18.056056 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:18.055964 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:18.201883 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:18.201854 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw"] Jul 23 15:03:18.202887 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:18.202863 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44926b7_ece0_4371_89be_a57a8f5cd019.slice/crio-0332d1bb992dd5214d25dac866ad2cb933c4c71458f18e52742d43f421c76283 WatchSource:0}: Error finding container 0332d1bb992dd5214d25dac866ad2cb933c4c71458f18e52742d43f421c76283: Status 404 returned error can't find the container with id 0332d1bb992dd5214d25dac866ad2cb933c4c71458f18e52742d43f421c76283 Jul 23 15:03:19.047536 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:19.047499 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" event={"ID":"a44926b7-ece0-4371-89be-a57a8f5cd019","Type":"ContainerStarted","Data":"0332d1bb992dd5214d25dac866ad2cb933c4c71458f18e52742d43f421c76283"} Jul 23 15:03:20.055022 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:20.054986 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" event={"ID":"a44926b7-ece0-4371-89be-a57a8f5cd019","Type":"ContainerStarted","Data":"73142f32e614f84bb9c573c642748089453bc859cf936aa0bcf12d83b957af09"} Jul 23 15:03:20.055436 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:20.055111 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:20.080587 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:20.080534 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" podStartSLOduration=1.393154204 podStartE2EDuration="3.080518135s" totalImagesPullingTime="1.687363931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:03:20.078286013 +0000 UTC m=+648.229735378" watchObservedRunningTime="2026-07-23 15:03:20.080518135 +0000 UTC m=+648.231967503" Jul 23 15:03:21.788065 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:21.788029 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz"] Jul 23 15:03:21.791294 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:21.791269 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:21.794621 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:21.794321 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gmqhn\"" Jul 23 15:03:21.805776 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:21.805746 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz"] Jul 23 15:03:21.911629 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:21.911600 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx4pr\" (UniqueName: \"kubernetes.io/projected/cc2286fa-1743-40a2-93a8-fa00ad471e92-kube-api-access-gx4pr\") pod \"authorino-operator-75cc8d5cb7-9xvpz\" (UID: \"cc2286fa-1743-40a2-93a8-fa00ad471e92\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:22.012100 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:22.012059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx4pr\" (UniqueName: \"kubernetes.io/projected/cc2286fa-1743-40a2-93a8-fa00ad471e92-kube-api-access-gx4pr\") pod \"authorino-operator-75cc8d5cb7-9xvpz\" (UID: \"cc2286fa-1743-40a2-93a8-fa00ad471e92\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:22.029145 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:22.029119 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx4pr\" (UniqueName: \"kubernetes.io/projected/cc2286fa-1743-40a2-93a8-fa00ad471e92-kube-api-access-gx4pr\") pod \"authorino-operator-75cc8d5cb7-9xvpz\" (UID: \"cc2286fa-1743-40a2-93a8-fa00ad471e92\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:22.111827 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:22.111746 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:22.251891 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:22.251865 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz"] Jul 23 15:03:22.253102 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:22.253075 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2286fa_1743_40a2_93a8_fa00ad471e92.slice/crio-095fc890c2c1e59e9e160c6f55db7d52213c837b2789a9a573af8aaea71a24c2 WatchSource:0}: Error finding container 095fc890c2c1e59e9e160c6f55db7d52213c837b2789a9a573af8aaea71a24c2: Status 404 returned error can't find the container with id 095fc890c2c1e59e9e160c6f55db7d52213c837b2789a9a573af8aaea71a24c2 Jul 23 15:03:23.074283 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:23.074215 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" event={"ID":"cc2286fa-1743-40a2-93a8-fa00ad471e92","Type":"ContainerStarted","Data":"095fc890c2c1e59e9e160c6f55db7d52213c837b2789a9a573af8aaea71a24c2"} Jul 23 15:03:26.093310 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.093250 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" event={"ID":"cc2286fa-1743-40a2-93a8-fa00ad471e92","Type":"ContainerStarted","Data":"47d831ff80eabd01548c51ab5ad4d1037573c39b64e789138cf2d6456cbf7e29"} Jul 23 15:03:26.093778 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.093326 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:26.112639 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.112583 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" podStartSLOduration=2.33223793 podStartE2EDuration="5.112567998s" totalImagesPullingTime="2.780330068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:03:26.111024143 +0000 UTC m=+654.262473512" watchObservedRunningTime="2026-07-23 15:03:26.112567998 +0000 UTC m=+654.264017366" Jul 23 15:03:26.746891 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.746855 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4"] Jul 23 15:03:26.749383 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.749365 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:26.752014 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.751998 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xsjqm\"" Jul 23 15:03:26.764303 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.764282 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4"] Jul 23 15:03:26.856746 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.856717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:26.856922 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.856768 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8q9l\" (UniqueName: \"kubernetes.io/projected/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-kube-api-access-c8q9l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:26.957484 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.957443 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:26.957630 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.957508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8q9l\" (UniqueName: \"kubernetes.io/projected/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-kube-api-access-c8q9l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:26.957843 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.957823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:26.969589 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:26.969566 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8q9l\" (UniqueName: \"kubernetes.io/projected/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-kube-api-access-c8q9l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:27.060803 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:27.060706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:27.214032 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:27.214006 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4"] Jul 23 15:03:27.215444 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:27.215415 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5ab54f_3c22_4d57_84c7_adb0e5a822e2.slice/crio-c127fe4d8a8efef7b9712620b7e399ccc77ee31e1ceceeb9841452e1dd2d4e93 WatchSource:0}: Error finding container c127fe4d8a8efef7b9712620b7e399ccc77ee31e1ceceeb9841452e1dd2d4e93: Status 404 returned error can't find the container with id c127fe4d8a8efef7b9712620b7e399ccc77ee31e1ceceeb9841452e1dd2d4e93 Jul 23 15:03:28.107119 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:28.107077 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" event={"ID":"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2","Type":"ContainerStarted","Data":"c127fe4d8a8efef7b9712620b7e399ccc77ee31e1ceceeb9841452e1dd2d4e93"} Jul 23 15:03:31.063047 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:31.063011 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:33.139355 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:33.139318 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" event={"ID":"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2","Type":"ContainerStarted","Data":"d893f44e56750ec35f9e2130961dff45020e225bbe634faa7efb26cbdf51d58e"} Jul 23 15:03:33.139738 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:33.139417 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:33.161343 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:33.161287 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" podStartSLOduration=1.752753945 podStartE2EDuration="7.161246961s" totalImagesPullingTime="5.408493016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:03:33.15959179 +0000 UTC m=+661.311041151" watchObservedRunningTime="2026-07-23 15:03:33.161246961 +0000 UTC m=+661.312696329" Jul 23 15:03:37.101111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:37.101079 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9xvpz" Jul 23 15:03:44.147519 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:44.147488 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:45.297310 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.297217 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p"] Jul 23 15:03:45.304020 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.303995 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.313588 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.313554 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p"] Jul 23 15:03:45.442310 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.442242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnxzv\" (UniqueName: \"kubernetes.io/projected/26bbbd74-a672-40ff-98d4-3f0ff7047649-kube-api-access-wnxzv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.442493 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.442403 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bbbd74-a672-40ff-98d4-3f0ff7047649-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.543045 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.543003 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnxzv\" (UniqueName: \"kubernetes.io/projected/26bbbd74-a672-40ff-98d4-3f0ff7047649-kube-api-access-wnxzv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.543241 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.543128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bbbd74-a672-40ff-98d4-3f0ff7047649-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.543541 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.543515 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bbbd74-a672-40ff-98d4-3f0ff7047649-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.553013 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.552952 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnxzv\" (UniqueName: \"kubernetes.io/projected/26bbbd74-a672-40ff-98d4-3f0ff7047649-kube-api-access-wnxzv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.615461 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.615427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:45.760135 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.760093 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p"] Jul 23 15:03:45.760480 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:45.760458 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bbbd74_a672_40ff_98d4_3f0ff7047649.slice/crio-c68e35af68cc212381fe6d5228e1245c369d657858d1f0c87723f21b3629c657 WatchSource:0}: Error finding container c68e35af68cc212381fe6d5228e1245c369d657858d1f0c87723f21b3629c657: Status 404 returned error can't find the container with id c68e35af68cc212381fe6d5228e1245c369d657858d1f0c87723f21b3629c657 Jul 23 15:03:45.997469 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:45.997434 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p"] Jul 23 15:03:46.003743 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.003716 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p"] Jul 23 15:03:46.015583 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.015558 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4"] Jul 23 15:03:46.015770 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.015727 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" containerName="manager" containerID="cri-o://d893f44e56750ec35f9e2130961dff45020e225bbe634faa7efb26cbdf51d58e" gracePeriod=2 Jul 23 15:03:46.027861 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.027834 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb"] Jul 23 15:03:46.029112 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.029005 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" containerName="manager" Jul 23 15:03:46.029112 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.029032 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" containerName="manager" Jul 23 15:03:46.029287 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.029186 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" containerName="manager" Jul 23 15:03:46.032608 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.032586 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4"] Jul 23 15:03:46.032721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.032707 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.039112 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.038936 2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\": pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.041407 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.041384 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l"] Jul 23 15:03:46.041778 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.041762 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" containerName="manager" Jul 23 15:03:46.041859 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.041781 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" containerName="manager" Jul 23 15:03:46.041913 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.041878 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" containerName="manager" Jul 23 15:03:46.045371 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.045352 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.045776 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.045758 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw"] Jul 23 15:03:46.045954 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.045919 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" containerName="manager" containerID="cri-o://73142f32e614f84bb9c573c642748089453bc859cf936aa0bcf12d83b957af09" gracePeriod=2 Jul 23 15:03:46.049318 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.049274 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb"] Jul 23 15:03:46.060688 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.060531 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l"] Jul 23 15:03:46.063309 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.063238 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw"] Jul 23 15:03:46.066104 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.066050 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.068420 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.068387 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.069672 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.069634 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf"] Jul 23 15:03:46.070158 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.070141 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" containerName="manager" Jul 23 15:03:46.070241 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.070162 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" containerName="manager" Jul 23 15:03:46.070327 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.070291 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" containerName="manager" Jul 23 15:03:46.073958 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.073936 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:46.083091 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.083069 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf"] Jul 23 15:03:46.091884 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.091852 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:46.111945 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.111911 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.114050 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.114014 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:46.148130 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.148105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdeb1ce4-e35c-4526-b728-6108b55287af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zrppb\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.148289 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.148142 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b091c48a-31c7-43bc-a18e-9ee48535eae6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w57l\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.148289 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.148184 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxgkg\" (UniqueName: \"kubernetes.io/projected/b091c48a-31c7-43bc-a18e-9ee48535eae6-kube-api-access-hxgkg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w57l\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.148383 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.148353 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzp9b\" (UniqueName: \"kubernetes.io/projected/bdeb1ce4-e35c-4526-b728-6108b55287af-kube-api-access-rzp9b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zrppb\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.213198 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.213165 2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" container="manager" Jul 23 15:03:46.215677 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.215522 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:46.215819 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.215699 2582 generic.go:356] "Generic (PLEG): container finished" podID="a44926b7-ece0-4371-89be-a57a8f5cd019" containerID="73142f32e614f84bb9c573c642748089453bc859cf936aa0bcf12d83b957af09" exitCode=0 Jul 23 15:03:46.217787 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.217645 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.218589 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.218563 2582 generic.go:356] "Generic (PLEG): container finished" podID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" containerID="d893f44e56750ec35f9e2130961dff45020e225bbe634faa7efb26cbdf51d58e" exitCode=0 Jul 23 15:03:46.219717 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.219675 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:46.249731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.249684 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdmz6\" (UniqueName: \"kubernetes.io/projected/f9dc316e-b910-495e-8c00-302b1bfe9700-kube-api-access-xdmz6\") pod \"limitador-operator-controller-manager-859d7b78bd-n94sf\" (UID: \"f9dc316e-b910-495e-8c00-302b1bfe9700\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:46.249919 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.249793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzp9b\" (UniqueName: \"kubernetes.io/projected/bdeb1ce4-e35c-4526-b728-6108b55287af-kube-api-access-rzp9b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zrppb\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.249919 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.249846 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdeb1ce4-e35c-4526-b728-6108b55287af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zrppb\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.249919 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.249884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b091c48a-31c7-43bc-a18e-9ee48535eae6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w57l\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.250093 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.249930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxgkg\" (UniqueName: \"kubernetes.io/projected/b091c48a-31c7-43bc-a18e-9ee48535eae6-kube-api-access-hxgkg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w57l\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.250344 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.250316 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b091c48a-31c7-43bc-a18e-9ee48535eae6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w57l\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.250448 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.250365 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdeb1ce4-e35c-4526-b728-6108b55287af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zrppb\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.259560 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.259528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxgkg\" (UniqueName: \"kubernetes.io/projected/b091c48a-31c7-43bc-a18e-9ee48535eae6-kube-api-access-hxgkg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9w57l\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.259689 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.259589 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzp9b\" (UniqueName: \"kubernetes.io/projected/bdeb1ce4-e35c-4526-b728-6108b55287af-kube-api-access-rzp9b\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-zrppb\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.280731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.280706 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.283339 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.283308 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:46.285555 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.285527 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.287503 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.287481 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:46.287642 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.287611 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:46.289791 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.289754 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:46.292365 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.291888 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhxd4\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:46.295672 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:46.294297 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rcmdw\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:46.351404 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.351309 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8q9l\" (UniqueName: \"kubernetes.io/projected/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-kube-api-access-c8q9l\") pod \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " Jul 23 15:03:46.351404 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.351370 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-extensions-socket-volume\") pod \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\" (UID: \"7d5ab54f-3c22-4d57-84c7-adb0e5a822e2\") " Jul 23 15:03:46.351844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.351414 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9jx2\" (UniqueName: \"kubernetes.io/projected/a44926b7-ece0-4371-89be-a57a8f5cd019-kube-api-access-p9jx2\") pod \"a44926b7-ece0-4371-89be-a57a8f5cd019\" (UID: \"a44926b7-ece0-4371-89be-a57a8f5cd019\") " Jul 23 15:03:46.351844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.351525 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdmz6\" (UniqueName: \"kubernetes.io/projected/f9dc316e-b910-495e-8c00-302b1bfe9700-kube-api-access-xdmz6\") pod \"limitador-operator-controller-manager-859d7b78bd-n94sf\" (UID: \"f9dc316e-b910-495e-8c00-302b1bfe9700\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:46.351934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.351902 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" (UID: "7d5ab54f-3c22-4d57-84c7-adb0e5a822e2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:03:46.353799 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.353775 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-kube-api-access-c8q9l" (OuterVolumeSpecName: "kube-api-access-c8q9l") pod "7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" (UID: "7d5ab54f-3c22-4d57-84c7-adb0e5a822e2"). InnerVolumeSpecName "kube-api-access-c8q9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:03:46.353872 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.353816 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a44926b7-ece0-4371-89be-a57a8f5cd019-kube-api-access-p9jx2" (OuterVolumeSpecName: "kube-api-access-p9jx2") pod "a44926b7-ece0-4371-89be-a57a8f5cd019" (UID: "a44926b7-ece0-4371-89be-a57a8f5cd019"). InnerVolumeSpecName "kube-api-access-p9jx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:03:46.368515 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.368484 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdmz6\" (UniqueName: \"kubernetes.io/projected/f9dc316e-b910-495e-8c00-302b1bfe9700-kube-api-access-xdmz6\") pod \"limitador-operator-controller-manager-859d7b78bd-n94sf\" (UID: \"f9dc316e-b910-495e-8c00-302b1bfe9700\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:46.442951 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.442913 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:46.450554 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.450356 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:46.452889 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.452849 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-extensions-socket-volume\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:46.452889 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.452886 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9jx2\" (UniqueName: \"kubernetes.io/projected/a44926b7-ece0-4371-89be-a57a8f5cd019-kube-api-access-p9jx2\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:46.453032 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.452901 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8q9l\" (UniqueName: \"kubernetes.io/projected/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2-kube-api-access-c8q9l\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:46.456501 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.456482 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:46.618172 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.618086 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb"] Jul 23 15:03:46.619822 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:46.619787 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdeb1ce4_e35c_4526_b728_6108b55287af.slice/crio-0db760d510d46bd5bf7307c8f15a56e77f19df48e8b51242dd6d103e0df00874 WatchSource:0}: Error finding container 0db760d510d46bd5bf7307c8f15a56e77f19df48e8b51242dd6d103e0df00874: Status 404 returned error can't find the container with id 0db760d510d46bd5bf7307c8f15a56e77f19df48e8b51242dd6d103e0df00874 Jul 23 15:03:46.633841 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.633810 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l"] Jul 23 15:03:46.636642 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:46.636613 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb091c48a_31c7_43bc_a18e_9ee48535eae6.slice/crio-41f44316eafc341b9e186cae330842a451f26346e0f3f4df10aadfa743b07a82 WatchSource:0}: Error finding container 41f44316eafc341b9e186cae330842a451f26346e0f3f4df10aadfa743b07a82: Status 404 returned error can't find the container with id 41f44316eafc341b9e186cae330842a451f26346e0f3f4df10aadfa743b07a82 Jul 23 15:03:46.655420 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:46.655397 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf"] Jul 23 15:03:46.658391 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:46.658365 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9dc316e_b910_495e_8c00_302b1bfe9700.slice/crio-3ebb7bf9f09e1cd45ca00f2478024d28970ed91a05628eea383166e946c465ee WatchSource:0}: Error finding container 3ebb7bf9f09e1cd45ca00f2478024d28970ed91a05628eea383166e946c465ee: Status 404 returned error can't find the container with id 3ebb7bf9f09e1cd45ca00f2478024d28970ed91a05628eea383166e946c465ee Jul 23 15:03:47.225567 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.225537 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rcmdw" Jul 23 15:03:47.225567 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.225549 2582 scope.go:117] "RemoveContainer" containerID="73142f32e614f84bb9c573c642748089453bc859cf936aa0bcf12d83b957af09" Jul 23 15:03:47.227453 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.227422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" event={"ID":"f9dc316e-b910-495e-8c00-302b1bfe9700","Type":"ContainerStarted","Data":"060f422015a401495566b73f3f89fea9224c848c94c4a188659102e4190204e8"} Jul 23 15:03:47.227584 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.227491 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:47.227584 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.227519 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" event={"ID":"f9dc316e-b910-495e-8c00-302b1bfe9700","Type":"ContainerStarted","Data":"3ebb7bf9f09e1cd45ca00f2478024d28970ed91a05628eea383166e946c465ee"} Jul 23 15:03:47.228914 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.228894 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhxd4" Jul 23 15:03:47.230800 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.230777 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" event={"ID":"bdeb1ce4-e35c-4526-b728-6108b55287af","Type":"ContainerStarted","Data":"3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e"} Jul 23 15:03:47.230917 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.230825 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" event={"ID":"bdeb1ce4-e35c-4526-b728-6108b55287af","Type":"ContainerStarted","Data":"0db760d510d46bd5bf7307c8f15a56e77f19df48e8b51242dd6d103e0df00874"} Jul 23 15:03:47.230917 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.230886 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:47.232761 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.232737 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" event={"ID":"b091c48a-31c7-43bc-a18e-9ee48535eae6","Type":"ContainerStarted","Data":"59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872"} Jul 23 15:03:47.232884 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.232783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" event={"ID":"b091c48a-31c7-43bc-a18e-9ee48535eae6","Type":"ContainerStarted","Data":"41f44316eafc341b9e186cae330842a451f26346e0f3f4df10aadfa743b07a82"} Jul 23 15:03:47.232884 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.232847 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:47.234914 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.234889 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" containerName="manager" containerID="cri-o://f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0" gracePeriod=2 Jul 23 15:03:47.236701 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.236680 2582 scope.go:117] "RemoveContainer" containerID="d893f44e56750ec35f9e2130961dff45020e225bbe634faa7efb26cbdf51d58e" Jul 23 15:03:47.252508 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.252460 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" podStartSLOduration=1.2524447269999999 podStartE2EDuration="1.252444727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:03:47.249741984 +0000 UTC m=+675.401191353" watchObservedRunningTime="2026-07-23 15:03:47.252444727 +0000 UTC m=+675.403894158" Jul 23 15:03:47.273475 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.273424 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" podStartSLOduration=1.273402355 podStartE2EDuration="1.273402355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:03:47.271880197 +0000 UTC m=+675.423329605" watchObservedRunningTime="2026-07-23 15:03:47.273402355 +0000 UTC m=+675.424851725" Jul 23 15:03:47.297660 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.297612 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" podStartSLOduration=1.29759735 podStartE2EDuration="1.29759735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:03:47.295725189 +0000 UTC m=+675.447174559" watchObservedRunningTime="2026-07-23 15:03:47.29759735 +0000 UTC m=+675.449046720" Jul 23 15:03:47.474313 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.474283 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:47.477239 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:47.477166 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:47.560250 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.560227 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bbbd74-a672-40ff-98d4-3f0ff7047649-extensions-socket-volume\") pod \"26bbbd74-a672-40ff-98d4-3f0ff7047649\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " Jul 23 15:03:47.560382 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.560320 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnxzv\" (UniqueName: \"kubernetes.io/projected/26bbbd74-a672-40ff-98d4-3f0ff7047649-kube-api-access-wnxzv\") pod \"26bbbd74-a672-40ff-98d4-3f0ff7047649\" (UID: \"26bbbd74-a672-40ff-98d4-3f0ff7047649\") " Jul 23 15:03:47.560507 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.560481 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26bbbd74-a672-40ff-98d4-3f0ff7047649-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "26bbbd74-a672-40ff-98d4-3f0ff7047649" (UID: "26bbbd74-a672-40ff-98d4-3f0ff7047649"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:03:47.560669 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.560624 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bbbd74-a672-40ff-98d4-3f0ff7047649-extensions-socket-volume\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:47.562669 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.562647 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26bbbd74-a672-40ff-98d4-3f0ff7047649-kube-api-access-wnxzv" (OuterVolumeSpecName: "kube-api-access-wnxzv") pod "26bbbd74-a672-40ff-98d4-3f0ff7047649" (UID: "26bbbd74-a672-40ff-98d4-3f0ff7047649"). InnerVolumeSpecName "kube-api-access-wnxzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:03:47.661468 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:47.661429 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnxzv\" (UniqueName: \"kubernetes.io/projected/26bbbd74-a672-40ff-98d4-3f0ff7047649-kube-api-access-wnxzv\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:48.245031 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.244999 2582 generic.go:356] "Generic (PLEG): container finished" podID="26bbbd74-a672-40ff-98d4-3f0ff7047649" containerID="f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0" exitCode=2 Jul 23 15:03:48.245215 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.245047 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:48.245215 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.245128 2582 scope.go:117] "RemoveContainer" containerID="f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0" Jul 23 15:03:48.247955 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:48.247926 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:48.256931 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.256911 2582 scope.go:117] "RemoveContainer" containerID="f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0" Jul 23 15:03:48.257051 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:48.256986 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-85r2p\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-85r2p" Jul 23 15:03:48.257518 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:48.257496 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0\": container with ID starting with f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0 not found: ID does not exist" containerID="f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0" Jul 23 15:03:48.257596 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.257525 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0"} err="failed to get container status \"f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0\": rpc error: code = NotFound desc = could not find container \"f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0\": container with ID starting with f7667319585cab327939d6e60cb3663256e24bea64078caa0823850f3a1dbdc0 not found: ID does not exist" Jul 23 15:03:48.294208 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.294183 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26bbbd74-a672-40ff-98d4-3f0ff7047649" path="/var/lib/kubelet/pods/26bbbd74-a672-40ff-98d4-3f0ff7047649/volumes" Jul 23 15:03:48.294549 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.294535 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d5ab54f-3c22-4d57-84c7-adb0e5a822e2" path="/var/lib/kubelet/pods/7d5ab54f-3c22-4d57-84c7-adb0e5a822e2/volumes" Jul 23 15:03:48.294812 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:48.294800 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a44926b7-ece0-4371-89be-a57a8f5cd019" path="/var/lib/kubelet/pods/a44926b7-ece0-4371-89be-a57a8f5cd019/volumes" Jul 23 15:03:58.247975 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.247947 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:03:58.248464 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.248024 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:58.248464 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.248161 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n94sf" Jul 23 15:03:58.317585 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.317547 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l"] Jul 23 15:03:58.317780 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.317734 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" podUID="b091c48a-31c7-43bc-a18e-9ee48535eae6" containerName="manager" containerID="cri-o://59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872" gracePeriod=10 Jul 23 15:03:58.576120 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.576100 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:58.660663 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.660636 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxgkg\" (UniqueName: \"kubernetes.io/projected/b091c48a-31c7-43bc-a18e-9ee48535eae6-kube-api-access-hxgkg\") pod \"b091c48a-31c7-43bc-a18e-9ee48535eae6\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " Jul 23 15:03:58.660862 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.660705 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b091c48a-31c7-43bc-a18e-9ee48535eae6-extensions-socket-volume\") pod \"b091c48a-31c7-43bc-a18e-9ee48535eae6\" (UID: \"b091c48a-31c7-43bc-a18e-9ee48535eae6\") " Jul 23 15:03:58.661145 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.661124 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b091c48a-31c7-43bc-a18e-9ee48535eae6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b091c48a-31c7-43bc-a18e-9ee48535eae6" (UID: "b091c48a-31c7-43bc-a18e-9ee48535eae6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:03:58.662856 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.662833 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b091c48a-31c7-43bc-a18e-9ee48535eae6-kube-api-access-hxgkg" (OuterVolumeSpecName: "kube-api-access-hxgkg") pod "b091c48a-31c7-43bc-a18e-9ee48535eae6" (UID: "b091c48a-31c7-43bc-a18e-9ee48535eae6"). InnerVolumeSpecName "kube-api-access-hxgkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:03:58.762014 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.761986 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b091c48a-31c7-43bc-a18e-9ee48535eae6-extensions-socket-volume\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:58.762014 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.762011 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxgkg\" (UniqueName: \"kubernetes.io/projected/b091c48a-31c7-43bc-a18e-9ee48535eae6-kube-api-access-hxgkg\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:03:58.862624 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.862517 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x"] Jul 23 15:03:58.862919 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.862905 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b091c48a-31c7-43bc-a18e-9ee48535eae6" containerName="manager" Jul 23 15:03:58.862919 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.862919 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b091c48a-31c7-43bc-a18e-9ee48535eae6" containerName="manager" Jul 23 15:03:58.863016 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.863007 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b091c48a-31c7-43bc-a18e-9ee48535eae6" containerName="manager" Jul 23 15:03:58.865096 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.865080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:58.879363 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.879340 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x"] Jul 23 15:03:58.963560 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.963521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2bd7\" (UniqueName: \"kubernetes.io/projected/667a7e00-a43f-4395-944a-30404fe7240d-kube-api-access-z2bd7\") pod \"kuadrant-operator-controller-manager-86878c769f-gbv4x\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:58.963728 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:58.963621 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/667a7e00-a43f-4395-944a-30404fe7240d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbv4x\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:59.064349 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.064308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/667a7e00-a43f-4395-944a-30404fe7240d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbv4x\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:59.064534 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.064375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bd7\" (UniqueName: \"kubernetes.io/projected/667a7e00-a43f-4395-944a-30404fe7240d-kube-api-access-z2bd7\") pod \"kuadrant-operator-controller-manager-86878c769f-gbv4x\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:59.064746 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.064725 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/667a7e00-a43f-4395-944a-30404fe7240d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbv4x\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:59.073657 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.073625 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2bd7\" (UniqueName: \"kubernetes.io/projected/667a7e00-a43f-4395-944a-30404fe7240d-kube-api-access-z2bd7\") pod \"kuadrant-operator-controller-manager-86878c769f-gbv4x\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:59.174852 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.174818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:03:59.313856 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.313816 2582 generic.go:356] "Generic (PLEG): container finished" podID="b091c48a-31c7-43bc-a18e-9ee48535eae6" containerID="59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872" exitCode=0 Jul 23 15:03:59.314138 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.313889 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" event={"ID":"b091c48a-31c7-43bc-a18e-9ee48535eae6","Type":"ContainerDied","Data":"59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872"} Jul 23 15:03:59.314138 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.313927 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" event={"ID":"b091c48a-31c7-43bc-a18e-9ee48535eae6","Type":"ContainerDied","Data":"41f44316eafc341b9e186cae330842a451f26346e0f3f4df10aadfa743b07a82"} Jul 23 15:03:59.314138 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.313930 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l" Jul 23 15:03:59.314138 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.313948 2582 scope.go:117] "RemoveContainer" containerID="59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872" Jul 23 15:03:59.315671 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.315650 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x"] Jul 23 15:03:59.316914 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:03:59.316887 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667a7e00_a43f_4395_944a_30404fe7240d.slice/crio-6ba2ebad94dfff96a1f41d792e2d16bbf8f9a7ace2d90a24e052c4e52a86dbb2 WatchSource:0}: Error finding container 6ba2ebad94dfff96a1f41d792e2d16bbf8f9a7ace2d90a24e052c4e52a86dbb2: Status 404 returned error can't find the container with id 6ba2ebad94dfff96a1f41d792e2d16bbf8f9a7ace2d90a24e052c4e52a86dbb2 Jul 23 15:03:59.323812 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.323797 2582 scope.go:117] "RemoveContainer" containerID="59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872" Jul 23 15:03:59.324076 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:03:59.324053 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872\": container with ID starting with 59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872 not found: ID does not exist" containerID="59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872" Jul 23 15:03:59.324164 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.324086 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872"} err="failed to get container status \"59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872\": rpc error: code = NotFound desc = could not find container \"59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872\": container with ID starting with 59f06f7c5048f3fe4241b0303b202a78bfddf7f4c79377081ae36bfad6b54872 not found: ID does not exist" Jul 23 15:03:59.340746 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.340710 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l"] Jul 23 15:03:59.351285 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:03:59.351238 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9w57l"] Jul 23 15:04:00.294635 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:00.294601 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b091c48a-31c7-43bc-a18e-9ee48535eae6" path="/var/lib/kubelet/pods/b091c48a-31c7-43bc-a18e-9ee48535eae6/volumes" Jul 23 15:04:00.321002 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:00.320965 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" event={"ID":"667a7e00-a43f-4395-944a-30404fe7240d","Type":"ContainerStarted","Data":"a479833984a0cb7c03c8569d831216a9e7b95ab699c1cf146417012cfbaeceef"} Jul 23 15:04:00.321461 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:00.321021 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" event={"ID":"667a7e00-a43f-4395-944a-30404fe7240d","Type":"ContainerStarted","Data":"6ba2ebad94dfff96a1f41d792e2d16bbf8f9a7ace2d90a24e052c4e52a86dbb2"} Jul 23 15:04:00.321461 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:00.321154 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:04:00.366924 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:00.366877 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" podStartSLOduration=2.3668619619999998 podStartE2EDuration="2.366861962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:04:00.363815437 +0000 UTC m=+688.515264806" watchObservedRunningTime="2026-07-23 15:04:00.366861962 +0000 UTC m=+688.518311331" Jul 23 15:04:11.329093 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.329060 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:04:11.384697 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.384667 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb"] Jul 23 15:04:11.384928 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.384905 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" podUID="bdeb1ce4-e35c-4526-b728-6108b55287af" containerName="manager" containerID="cri-o://3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e" gracePeriod=10 Jul 23 15:04:11.633679 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.633658 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:04:11.778536 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.778504 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdeb1ce4-e35c-4526-b728-6108b55287af-extensions-socket-volume\") pod \"bdeb1ce4-e35c-4526-b728-6108b55287af\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " Jul 23 15:04:11.778741 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.778573 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzp9b\" (UniqueName: \"kubernetes.io/projected/bdeb1ce4-e35c-4526-b728-6108b55287af-kube-api-access-rzp9b\") pod \"bdeb1ce4-e35c-4526-b728-6108b55287af\" (UID: \"bdeb1ce4-e35c-4526-b728-6108b55287af\") " Jul 23 15:04:11.778942 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.778919 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeb1ce4-e35c-4526-b728-6108b55287af-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bdeb1ce4-e35c-4526-b728-6108b55287af" (UID: "bdeb1ce4-e35c-4526-b728-6108b55287af"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:04:11.780873 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.780849 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdeb1ce4-e35c-4526-b728-6108b55287af-kube-api-access-rzp9b" (OuterVolumeSpecName: "kube-api-access-rzp9b") pod "bdeb1ce4-e35c-4526-b728-6108b55287af" (UID: "bdeb1ce4-e35c-4526-b728-6108b55287af"). InnerVolumeSpecName "kube-api-access-rzp9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:04:11.879513 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.879421 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzp9b\" (UniqueName: \"kubernetes.io/projected/bdeb1ce4-e35c-4526-b728-6108b55287af-kube-api-access-rzp9b\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:04:11.879513 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:11.879457 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdeb1ce4-e35c-4526-b728-6108b55287af-extensions-socket-volume\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:04:12.396089 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.396057 2582 generic.go:356] "Generic (PLEG): container finished" podID="bdeb1ce4-e35c-4526-b728-6108b55287af" containerID="3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e" exitCode=0 Jul 23 15:04:12.396617 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.396102 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" Jul 23 15:04:12.396617 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.396149 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" event={"ID":"bdeb1ce4-e35c-4526-b728-6108b55287af","Type":"ContainerDied","Data":"3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e"} Jul 23 15:04:12.396617 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.396207 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb" event={"ID":"bdeb1ce4-e35c-4526-b728-6108b55287af","Type":"ContainerDied","Data":"0db760d510d46bd5bf7307c8f15a56e77f19df48e8b51242dd6d103e0df00874"} Jul 23 15:04:12.396617 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.396241 2582 scope.go:117] "RemoveContainer" containerID="3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e" Jul 23 15:04:12.405881 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.405860 2582 scope.go:117] "RemoveContainer" containerID="3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e" Jul 23 15:04:12.406139 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:04:12.406110 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e\": container with ID starting with 3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e not found: ID does not exist" containerID="3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e" Jul 23 15:04:12.406271 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.406147 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e"} err="failed to get container status \"3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e\": rpc error: code = NotFound desc = could not find container \"3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e\": container with ID starting with 3dbc48365ebc38defb78331e9b1c2e8e2053466f0f27b4c3835a08bcc052cf9e not found: ID does not exist" Jul 23 15:04:12.425024 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.424997 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb"] Jul 23 15:04:12.429963 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:12.429939 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-zrppb"] Jul 23 15:04:14.295128 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:14.295092 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdeb1ce4-e35c-4526-b728-6108b55287af" path="/var/lib/kubelet/pods/bdeb1ce4-e35c-4526-b728-6108b55287af/volumes" Jul 23 15:04:30.562946 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.562861 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:30.563440 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.563338 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdeb1ce4-e35c-4526-b728-6108b55287af" containerName="manager" Jul 23 15:04:30.563440 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.563353 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdeb1ce4-e35c-4526-b728-6108b55287af" containerName="manager" Jul 23 15:04:30.563440 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.563418 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdeb1ce4-e35c-4526-b728-6108b55287af" containerName="manager" Jul 23 15:04:30.565569 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.565554 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.568520 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.568494 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 15:04:30.568642 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.568536 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f9sh9\"" Jul 23 15:04:30.574325 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.574300 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:30.659342 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.659308 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:30.747348 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.747312 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bds9c\" (UniqueName: \"kubernetes.io/projected/d3d6a322-a5d9-46b2-ae0a-907325adac59-kube-api-access-bds9c\") pod \"limitador-limitador-64c94fd64f-phmg8\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.747512 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.747393 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d3d6a322-a5d9-46b2-ae0a-907325adac59-config-file\") pod \"limitador-limitador-64c94fd64f-phmg8\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.848975 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.848892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bds9c\" (UniqueName: \"kubernetes.io/projected/d3d6a322-a5d9-46b2-ae0a-907325adac59-kube-api-access-bds9c\") pod \"limitador-limitador-64c94fd64f-phmg8\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.848975 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.848949 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d3d6a322-a5d9-46b2-ae0a-907325adac59-config-file\") pod \"limitador-limitador-64c94fd64f-phmg8\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.849552 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.849535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d3d6a322-a5d9-46b2-ae0a-907325adac59-config-file\") pod \"limitador-limitador-64c94fd64f-phmg8\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.858561 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.858528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bds9c\" (UniqueName: \"kubernetes.io/projected/d3d6a322-a5d9-46b2-ae0a-907325adac59-kube-api-access-bds9c\") pod \"limitador-limitador-64c94fd64f-phmg8\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:30.876274 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:30.876233 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:31.021572 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.021537 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:31.022864 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:04:31.022834 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d6a322_a5d9_46b2_ae0a_907325adac59.slice/crio-1ad89acffa280a375461ada4fc80f4428ccd01f3e8bc5ff7f2228338e65ace6b WatchSource:0}: Error finding container 1ad89acffa280a375461ada4fc80f4428ccd01f3e8bc5ff7f2228338e65ace6b: Status 404 returned error can't find the container with id 1ad89acffa280a375461ada4fc80f4428ccd01f3e8bc5ff7f2228338e65ace6b Jul 23 15:04:31.359280 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.359231 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-s4ctb"] Jul 23 15:04:31.362640 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.362616 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:31.365387 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.365360 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cn46q\"" Jul 23 15:04:31.369360 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.369325 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s4ctb"] Jul 23 15:04:31.454537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.454498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnhqf\" (UniqueName: \"kubernetes.io/projected/c248dd58-3c05-48a2-9b1d-e903497391ba-kube-api-access-cnhqf\") pod \"authorino-58cd955675-s4ctb\" (UID: \"c248dd58-3c05-48a2-9b1d-e903497391ba\") " pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:31.468086 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.468053 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-xmj2c"] Jul 23 15:04:31.470473 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.470453 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:04:31.478951 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.478928 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xmj2c"] Jul 23 15:04:31.507137 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.507102 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" event={"ID":"d3d6a322-a5d9-46b2-ae0a-907325adac59","Type":"ContainerStarted","Data":"1ad89acffa280a375461ada4fc80f4428ccd01f3e8bc5ff7f2228338e65ace6b"} Jul 23 15:04:31.555697 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.555645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnhqf\" (UniqueName: \"kubernetes.io/projected/c248dd58-3c05-48a2-9b1d-e903497391ba-kube-api-access-cnhqf\") pod \"authorino-58cd955675-s4ctb\" (UID: \"c248dd58-3c05-48a2-9b1d-e903497391ba\") " pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:31.563821 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.563785 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnhqf\" (UniqueName: \"kubernetes.io/projected/c248dd58-3c05-48a2-9b1d-e903497391ba-kube-api-access-cnhqf\") pod \"authorino-58cd955675-s4ctb\" (UID: \"c248dd58-3c05-48a2-9b1d-e903497391ba\") " pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:31.656330 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.656298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plnmx\" (UniqueName: \"kubernetes.io/projected/cd3a62b1-def7-4d84-9a62-8c244f6d00fa-kube-api-access-plnmx\") pod \"authorino-75d556bd77-xmj2c\" (UID: \"cd3a62b1-def7-4d84-9a62-8c244f6d00fa\") " pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:04:31.673441 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.673413 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:31.756826 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.756798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plnmx\" (UniqueName: \"kubernetes.io/projected/cd3a62b1-def7-4d84-9a62-8c244f6d00fa-kube-api-access-plnmx\") pod \"authorino-75d556bd77-xmj2c\" (UID: \"cd3a62b1-def7-4d84-9a62-8c244f6d00fa\") " pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:04:31.765438 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.765408 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plnmx\" (UniqueName: \"kubernetes.io/projected/cd3a62b1-def7-4d84-9a62-8c244f6d00fa-kube-api-access-plnmx\") pod \"authorino-75d556bd77-xmj2c\" (UID: \"cd3a62b1-def7-4d84-9a62-8c244f6d00fa\") " pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:04:31.780181 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.780151 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:04:31.844148 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.844099 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s4ctb"] Jul 23 15:04:31.923550 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:31.923526 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xmj2c"] Jul 23 15:04:31.925593 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:04:31.925556 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd3a62b1_def7_4d84_9a62_8c244f6d00fa.slice/crio-dd2dbcd9ae5440dab22b85eeee9ad01b796cedc2e4795067197e007c5cee1e10 WatchSource:0}: Error finding container dd2dbcd9ae5440dab22b85eeee9ad01b796cedc2e4795067197e007c5cee1e10: Status 404 returned error can't find the container with id dd2dbcd9ae5440dab22b85eeee9ad01b796cedc2e4795067197e007c5cee1e10 Jul 23 15:04:32.513306 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:32.513246 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s4ctb" event={"ID":"c248dd58-3c05-48a2-9b1d-e903497391ba","Type":"ContainerStarted","Data":"41b61d8ef600040c5b624abcc30849557a426cc45aac9794f096667969258e2c"} Jul 23 15:04:32.514437 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:32.514406 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xmj2c" event={"ID":"cd3a62b1-def7-4d84-9a62-8c244f6d00fa","Type":"ContainerStarted","Data":"dd2dbcd9ae5440dab22b85eeee9ad01b796cedc2e4795067197e007c5cee1e10"} Jul 23 15:04:37.555292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.555175 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" event={"ID":"d3d6a322-a5d9-46b2-ae0a-907325adac59","Type":"ContainerStarted","Data":"5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002"} Jul 23 15:04:37.555292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.555238 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:37.556801 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.556774 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s4ctb" event={"ID":"c248dd58-3c05-48a2-9b1d-e903497391ba","Type":"ContainerStarted","Data":"62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a"} Jul 23 15:04:37.558100 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.558071 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xmj2c" event={"ID":"cd3a62b1-def7-4d84-9a62-8c244f6d00fa","Type":"ContainerStarted","Data":"9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c"} Jul 23 15:04:37.575443 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.575397 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" podStartSLOduration=1.414979768 podStartE2EDuration="7.575386253s" totalImagesPullingTime="6.160406485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:04:37.573035546 +0000 UTC m=+725.724484915" watchObservedRunningTime="2026-07-23 15:04:37.575386253 +0000 UTC m=+725.726835622" Jul 23 15:04:37.589702 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.589659 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-s4ctb" podStartSLOduration=1.237499192 podStartE2EDuration="6.589646101s" totalImagesPullingTime="5.352146909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:04:37.587283058 +0000 UTC m=+725.738732431" watchObservedRunningTime="2026-07-23 15:04:37.589646101 +0000 UTC m=+725.741095530" Jul 23 15:04:37.603041 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.602997 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-xmj2c" podStartSLOduration=1.342141139 podStartE2EDuration="6.602980537s" totalImagesPullingTime="5.260839398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:04:37.601164241 +0000 UTC m=+725.752613611" watchObservedRunningTime="2026-07-23 15:04:37.602980537 +0000 UTC m=+725.754429904" Jul 23 15:04:37.632855 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:37.632823 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s4ctb"] Jul 23 15:04:39.569166 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:39.569124 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-s4ctb" podUID="c248dd58-3c05-48a2-9b1d-e903497391ba" containerName="authorino" containerID="cri-o://62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a" gracePeriod=30 Jul 23 15:04:39.818565 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:39.818538 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:39.832157 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:39.832077 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnhqf\" (UniqueName: \"kubernetes.io/projected/c248dd58-3c05-48a2-9b1d-e903497391ba-kube-api-access-cnhqf\") pod \"c248dd58-3c05-48a2-9b1d-e903497391ba\" (UID: \"c248dd58-3c05-48a2-9b1d-e903497391ba\") " Jul 23 15:04:39.834659 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:39.834616 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c248dd58-3c05-48a2-9b1d-e903497391ba-kube-api-access-cnhqf" (OuterVolumeSpecName: "kube-api-access-cnhqf") pod "c248dd58-3c05-48a2-9b1d-e903497391ba" (UID: "c248dd58-3c05-48a2-9b1d-e903497391ba"). InnerVolumeSpecName "kube-api-access-cnhqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:04:39.933431 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:39.933389 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnhqf\" (UniqueName: \"kubernetes.io/projected/c248dd58-3c05-48a2-9b1d-e903497391ba-kube-api-access-cnhqf\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:04:40.575850 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.575760 2582 generic.go:356] "Generic (PLEG): container finished" podID="c248dd58-3c05-48a2-9b1d-e903497391ba" containerID="62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a" exitCode=0 Jul 23 15:04:40.575850 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.575805 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s4ctb" Jul 23 15:04:40.576292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.575803 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s4ctb" event={"ID":"c248dd58-3c05-48a2-9b1d-e903497391ba","Type":"ContainerDied","Data":"62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a"} Jul 23 15:04:40.576292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.575925 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s4ctb" event={"ID":"c248dd58-3c05-48a2-9b1d-e903497391ba","Type":"ContainerDied","Data":"41b61d8ef600040c5b624abcc30849557a426cc45aac9794f096667969258e2c"} Jul 23 15:04:40.576292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.575950 2582 scope.go:117] "RemoveContainer" containerID="62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a" Jul 23 15:04:40.585242 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.585220 2582 scope.go:117] "RemoveContainer" containerID="62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a" Jul 23 15:04:40.585567 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:04:40.585539 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a\": container with ID starting with 62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a not found: ID does not exist" containerID="62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a" Jul 23 15:04:40.585679 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.585573 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a"} err="failed to get container status \"62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a\": rpc error: code = NotFound desc = could not find container \"62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a\": container with ID starting with 62ea266d168943b627766baeb2b5c11486917b7846bf09b08c741952d8bc326a not found: ID does not exist" Jul 23 15:04:40.594347 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.594319 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s4ctb"] Jul 23 15:04:40.597916 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:40.597891 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-s4ctb"] Jul 23 15:04:42.293982 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:42.293948 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c248dd58-3c05-48a2-9b1d-e903497391ba" path="/var/lib/kubelet/pods/c248dd58-3c05-48a2-9b1d-e903497391ba/volumes" Jul 23 15:04:45.920893 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:45.920814 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:45.921393 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:45.921107 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" podUID="d3d6a322-a5d9-46b2-ae0a-907325adac59" containerName="limitador" containerID="cri-o://5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002" gracePeriod=30 Jul 23 15:04:45.923616 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:45.923593 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:46.486125 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.486065 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:46.591982 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.591943 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bds9c\" (UniqueName: \"kubernetes.io/projected/d3d6a322-a5d9-46b2-ae0a-907325adac59-kube-api-access-bds9c\") pod \"d3d6a322-a5d9-46b2-ae0a-907325adac59\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " Jul 23 15:04:46.592182 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.592008 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d3d6a322-a5d9-46b2-ae0a-907325adac59-config-file\") pod \"d3d6a322-a5d9-46b2-ae0a-907325adac59\" (UID: \"d3d6a322-a5d9-46b2-ae0a-907325adac59\") " Jul 23 15:04:46.592432 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.592405 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3d6a322-a5d9-46b2-ae0a-907325adac59-config-file" (OuterVolumeSpecName: "config-file") pod "d3d6a322-a5d9-46b2-ae0a-907325adac59" (UID: "d3d6a322-a5d9-46b2-ae0a-907325adac59"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:04:46.594270 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.594234 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3d6a322-a5d9-46b2-ae0a-907325adac59-kube-api-access-bds9c" (OuterVolumeSpecName: "kube-api-access-bds9c") pod "d3d6a322-a5d9-46b2-ae0a-907325adac59" (UID: "d3d6a322-a5d9-46b2-ae0a-907325adac59"). InnerVolumeSpecName "kube-api-access-bds9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:04:46.610503 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.610477 2582 generic.go:356] "Generic (PLEG): container finished" podID="d3d6a322-a5d9-46b2-ae0a-907325adac59" containerID="5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002" exitCode=0 Jul 23 15:04:46.610634 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.610522 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" Jul 23 15:04:46.610634 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.610564 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" event={"ID":"d3d6a322-a5d9-46b2-ae0a-907325adac59","Type":"ContainerDied","Data":"5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002"} Jul 23 15:04:46.610634 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.610624 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phmg8" event={"ID":"d3d6a322-a5d9-46b2-ae0a-907325adac59","Type":"ContainerDied","Data":"1ad89acffa280a375461ada4fc80f4428ccd01f3e8bc5ff7f2228338e65ace6b"} Jul 23 15:04:46.610796 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.610654 2582 scope.go:117] "RemoveContainer" containerID="5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002" Jul 23 15:04:46.620182 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.620165 2582 scope.go:117] "RemoveContainer" containerID="5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002" Jul 23 15:04:46.620490 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:04:46.620472 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002\": container with ID starting with 5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002 not found: ID does not exist" containerID="5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002" Jul 23 15:04:46.620561 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.620499 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002"} err="failed to get container status \"5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002\": rpc error: code = NotFound desc = could not find container \"5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002\": container with ID starting with 5394f69a183327b024fb47b41c549f5244f7a59683028adea44c4acb29a50002 not found: ID does not exist" Jul 23 15:04:46.633028 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.633002 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:46.635095 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.635075 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phmg8"] Jul 23 15:04:46.692923 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.692891 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bds9c\" (UniqueName: \"kubernetes.io/projected/d3d6a322-a5d9-46b2-ae0a-907325adac59-kube-api-access-bds9c\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:04:46.692923 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:46.692920 2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d3d6a322-a5d9-46b2-ae0a-907325adac59-config-file\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:04:48.294492 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:48.294457 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3d6a322-a5d9-46b2-ae0a-907325adac59" path="/var/lib/kubelet/pods/d3d6a322-a5d9-46b2-ae0a-907325adac59/volumes" Jul 23 15:04:49.274968 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.274924 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c48jr"] Jul 23 15:04:49.275345 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.275328 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c248dd58-3c05-48a2-9b1d-e903497391ba" containerName="authorino" Jul 23 15:04:49.275345 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.275343 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c248dd58-3c05-48a2-9b1d-e903497391ba" containerName="authorino" Jul 23 15:04:49.275521 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.275360 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3d6a322-a5d9-46b2-ae0a-907325adac59" containerName="limitador" Jul 23 15:04:49.275521 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.275365 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3d6a322-a5d9-46b2-ae0a-907325adac59" containerName="limitador" Jul 23 15:04:49.275521 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.275431 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3d6a322-a5d9-46b2-ae0a-907325adac59" containerName="limitador" Jul 23 15:04:49.275521 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.275443 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c248dd58-3c05-48a2-9b1d-e903497391ba" containerName="authorino" Jul 23 15:04:49.278225 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.278203 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.281641 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.281614 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 15:04:49.281815 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.281771 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 23 15:04:49.282246 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.282226 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 15:04:49.282933 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.282915 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-7nd26\"" Jul 23 15:04:49.289192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.289163 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c48jr"] Jul 23 15:04:49.317201 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.317168 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5559df12-cb1f-4743-9d86-ab3dfa908ab1-data\") pod \"postgres-69fb4594cf-c48jr\" (UID: \"5559df12-cb1f-4743-9d86-ab3dfa908ab1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.317614 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.317207 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5bb\" (UniqueName: \"kubernetes.io/projected/5559df12-cb1f-4743-9d86-ab3dfa908ab1-kube-api-access-vh5bb\") pod \"postgres-69fb4594cf-c48jr\" (UID: \"5559df12-cb1f-4743-9d86-ab3dfa908ab1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.418422 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.418375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5559df12-cb1f-4743-9d86-ab3dfa908ab1-data\") pod \"postgres-69fb4594cf-c48jr\" (UID: \"5559df12-cb1f-4743-9d86-ab3dfa908ab1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.418422 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.418426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5bb\" (UniqueName: \"kubernetes.io/projected/5559df12-cb1f-4743-9d86-ab3dfa908ab1-kube-api-access-vh5bb\") pod \"postgres-69fb4594cf-c48jr\" (UID: \"5559df12-cb1f-4743-9d86-ab3dfa908ab1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.418771 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.418749 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5559df12-cb1f-4743-9d86-ab3dfa908ab1-data\") pod \"postgres-69fb4594cf-c48jr\" (UID: \"5559df12-cb1f-4743-9d86-ab3dfa908ab1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.428651 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.428626 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5bb\" (UniqueName: \"kubernetes.io/projected/5559df12-cb1f-4743-9d86-ab3dfa908ab1-kube-api-access-vh5bb\") pod \"postgres-69fb4594cf-c48jr\" (UID: \"5559df12-cb1f-4743-9d86-ab3dfa908ab1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.590879 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.590781 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:49.736830 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:49.736802 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c48jr"] Jul 23 15:04:49.740013 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:04:49.739374 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5559df12_cb1f_4743_9d86_ab3dfa908ab1.slice/crio-2e5b44772eda7b85f083a9f35cffd57a091d4501c10d68909350ea69a3430ebd WatchSource:0}: Error finding container 2e5b44772eda7b85f083a9f35cffd57a091d4501c10d68909350ea69a3430ebd: Status 404 returned error can't find the container with id 2e5b44772eda7b85f083a9f35cffd57a091d4501c10d68909350ea69a3430ebd Jul 23 15:04:50.633639 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:50.633600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" event={"ID":"5559df12-cb1f-4743-9d86-ab3dfa908ab1","Type":"ContainerStarted","Data":"2e5b44772eda7b85f083a9f35cffd57a091d4501c10d68909350ea69a3430ebd"} Jul 23 15:04:55.668089 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:55.668049 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" event={"ID":"5559df12-cb1f-4743-9d86-ab3dfa908ab1","Type":"ContainerStarted","Data":"6aaed9388185d435d8d08b50cd564caf33bfcf76ebd298267cfe4d1c71544075"} Jul 23 15:04:55.668545 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:55.668193 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:04:55.686946 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:04:55.686868 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" podStartSLOduration=1.201713678 podStartE2EDuration="6.686853108s" totalImagesPullingTime="5.48513943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:04:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:04:55.685781398 +0000 UTC m=+743.837230757" watchObservedRunningTime="2026-07-23 15:04:55.686853108 +0000 UTC m=+743.838302476" Jul 23 15:05:01.703682 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:01.703652 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c48jr" Jul 23 15:05:02.609080 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.609036 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cbdc6df48-4pnzl"] Jul 23 15:05:02.615735 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.615710 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.618510 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.618487 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 15:05:02.623387 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.622990 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cbdc6df48-4pnzl"] Jul 23 15:05:02.743376 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.743321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crlf\" (UniqueName: \"kubernetes.io/projected/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-kube-api-access-9crlf\") pod \"authorino-5cbdc6df48-4pnzl\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.743768 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.743406 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-tls-cert\") pod \"authorino-5cbdc6df48-4pnzl\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.844423 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.844387 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crlf\" (UniqueName: \"kubernetes.io/projected/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-kube-api-access-9crlf\") pod \"authorino-5cbdc6df48-4pnzl\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.844604 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.844449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-tls-cert\") pod \"authorino-5cbdc6df48-4pnzl\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.847010 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.846989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-tls-cert\") pod \"authorino-5cbdc6df48-4pnzl\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.854527 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.854505 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crlf\" (UniqueName: \"kubernetes.io/projected/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-kube-api-access-9crlf\") pod \"authorino-5cbdc6df48-4pnzl\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:02.926362 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:02.926326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:05:03.069337 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:03.069308 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cbdc6df48-4pnzl"] Jul 23 15:05:03.070652 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:05:03.070622 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ff28b92_0f8a_43ce_be6a_61bc069f21b3.slice/crio-2b6f4d49724c93efad279c896230c165c736e717ca730ad4a149888faf44417a WatchSource:0}: Error finding container 2b6f4d49724c93efad279c896230c165c736e717ca730ad4a149888faf44417a: Status 404 returned error can't find the container with id 2b6f4d49724c93efad279c896230c165c736e717ca730ad4a149888faf44417a Jul 23 15:05:03.714151 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:03.714111 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" event={"ID":"6ff28b92-0f8a-43ce-be6a-61bc069f21b3","Type":"ContainerStarted","Data":"2b6f4d49724c93efad279c896230c165c736e717ca730ad4a149888faf44417a"} Jul 23 15:05:04.727862 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:04.727828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" event={"ID":"6ff28b92-0f8a-43ce-be6a-61bc069f21b3","Type":"ContainerStarted","Data":"2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7"} Jul 23 15:05:04.747845 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:04.747791 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" podStartSLOduration=1.748062337 podStartE2EDuration="2.74777235s" totalImagesPullingTime="999.710013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:05:04.745462914 +0000 UTC m=+752.896912282" watchObservedRunningTime="2026-07-23 15:05:04.74777235 +0000 UTC m=+752.899221721" Jul 23 15:05:04.775844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:04.775808 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xmj2c"] Jul 23 15:05:04.776045 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:04.776010 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-xmj2c" podUID="cd3a62b1-def7-4d84-9a62-8c244f6d00fa" containerName="authorino" containerID="cri-o://9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c" gracePeriod=30 Jul 23 15:05:05.029409 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.029381 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:05:05.164222 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.164189 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plnmx\" (UniqueName: \"kubernetes.io/projected/cd3a62b1-def7-4d84-9a62-8c244f6d00fa-kube-api-access-plnmx\") pod \"cd3a62b1-def7-4d84-9a62-8c244f6d00fa\" (UID: \"cd3a62b1-def7-4d84-9a62-8c244f6d00fa\") " Jul 23 15:05:05.166519 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.166489 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd3a62b1-def7-4d84-9a62-8c244f6d00fa-kube-api-access-plnmx" (OuterVolumeSpecName: "kube-api-access-plnmx") pod "cd3a62b1-def7-4d84-9a62-8c244f6d00fa" (UID: "cd3a62b1-def7-4d84-9a62-8c244f6d00fa"). InnerVolumeSpecName "kube-api-access-plnmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:05:05.265306 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.265187 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plnmx\" (UniqueName: \"kubernetes.io/projected/cd3a62b1-def7-4d84-9a62-8c244f6d00fa-kube-api-access-plnmx\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:05:05.734613 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.734580 2582 generic.go:356] "Generic (PLEG): container finished" podID="cd3a62b1-def7-4d84-9a62-8c244f6d00fa" containerID="9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c" exitCode=0 Jul 23 15:05:05.735045 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.734627 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-xmj2c" Jul 23 15:05:05.735045 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.734666 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xmj2c" event={"ID":"cd3a62b1-def7-4d84-9a62-8c244f6d00fa","Type":"ContainerDied","Data":"9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c"} Jul 23 15:05:05.735045 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.734708 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-xmj2c" event={"ID":"cd3a62b1-def7-4d84-9a62-8c244f6d00fa","Type":"ContainerDied","Data":"dd2dbcd9ae5440dab22b85eeee9ad01b796cedc2e4795067197e007c5cee1e10"} Jul 23 15:05:05.735045 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.734729 2582 scope.go:117] "RemoveContainer" containerID="9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c" Jul 23 15:05:05.746404 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.746375 2582 scope.go:117] "RemoveContainer" containerID="9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c" Jul 23 15:05:05.746744 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:05:05.746725 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c\": container with ID starting with 9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c not found: ID does not exist" containerID="9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c" Jul 23 15:05:05.746824 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.746754 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c"} err="failed to get container status \"9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c\": rpc error: code = NotFound desc = could not find container \"9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c\": container with ID starting with 9a1c679f2312685d197a59298e3165bdaa60115b91b87607059cbb481667699c not found: ID does not exist" Jul 23 15:05:05.758859 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.758819 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xmj2c"] Jul 23 15:05:05.762936 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:05.762913 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-xmj2c"] Jul 23 15:05:06.294075 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:06.294044 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd3a62b1-def7-4d84-9a62-8c244f6d00fa" path="/var/lib/kubelet/pods/cd3a62b1-def7-4d84-9a62-8c244f6d00fa/volumes" Jul 23 15:05:09.901602 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:09.901564 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859d6f66cd-bxz96"] Jul 23 15:05:09.902160 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:09.902145 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd3a62b1-def7-4d84-9a62-8c244f6d00fa" containerName="authorino" Jul 23 15:05:09.902212 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:09.902164 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd3a62b1-def7-4d84-9a62-8c244f6d00fa" containerName="authorino" Jul 23 15:05:09.902307 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:09.902295 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd3a62b1-def7-4d84-9a62-8c244f6d00fa" containerName="authorino" Jul 23 15:05:10.047486 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.047455 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859d6f66cd-bxz96"] Jul 23 15:05:10.047663 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.047556 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.051547 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.051520 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 15:05:10.051547 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.051521 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 15:05:10.051777 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.051521 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j8sc5\"" Jul 23 15:05:10.089946 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.089916 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-765b7646dc-6bz67"] Jul 23 15:05:10.123502 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.123466 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765b7646dc-6bz67"] Jul 23 15:05:10.123656 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.123615 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.210324 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.210206 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sfxc\" (UniqueName: \"kubernetes.io/projected/4256b057-cb0d-4d5a-91c0-f08a8a400d03-kube-api-access-6sfxc\") pod \"maas-controller-765b7646dc-6bz67\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.210324 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.210292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4256b057-cb0d-4d5a-91c0-f08a8a400d03-webhook-cert\") pod \"maas-controller-765b7646dc-6bz67\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.210523 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.210350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc0bce6f-f74a-4e21-9921-a52bd3c30892-webhook-cert\") pod \"maas-controller-859d6f66cd-bxz96\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.210523 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.210384 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tnkc\" (UniqueName: \"kubernetes.io/projected/cc0bce6f-f74a-4e21-9921-a52bd3c30892-kube-api-access-4tnkc\") pod \"maas-controller-859d6f66cd-bxz96\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.311315 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.311282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tnkc\" (UniqueName: \"kubernetes.io/projected/cc0bce6f-f74a-4e21-9921-a52bd3c30892-kube-api-access-4tnkc\") pod \"maas-controller-859d6f66cd-bxz96\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.311478 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.311346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sfxc\" (UniqueName: \"kubernetes.io/projected/4256b057-cb0d-4d5a-91c0-f08a8a400d03-kube-api-access-6sfxc\") pod \"maas-controller-765b7646dc-6bz67\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.311478 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.311397 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4256b057-cb0d-4d5a-91c0-f08a8a400d03-webhook-cert\") pod \"maas-controller-765b7646dc-6bz67\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.311478 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.311450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc0bce6f-f74a-4e21-9921-a52bd3c30892-webhook-cert\") pod \"maas-controller-859d6f66cd-bxz96\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.313971 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.313948 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc0bce6f-f74a-4e21-9921-a52bd3c30892-webhook-cert\") pod \"maas-controller-859d6f66cd-bxz96\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.319320 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.314649 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4256b057-cb0d-4d5a-91c0-f08a8a400d03-webhook-cert\") pod \"maas-controller-765b7646dc-6bz67\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.324081 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.324049 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sfxc\" (UniqueName: \"kubernetes.io/projected/4256b057-cb0d-4d5a-91c0-f08a8a400d03-kube-api-access-6sfxc\") pod \"maas-controller-765b7646dc-6bz67\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.324206 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.324144 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tnkc\" (UniqueName: \"kubernetes.io/projected/cc0bce6f-f74a-4e21-9921-a52bd3c30892-kube-api-access-4tnkc\") pod \"maas-controller-859d6f66cd-bxz96\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.358089 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.358060 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:10.434589 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.434522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:10.535723 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.535695 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859d6f66cd-bxz96"] Jul 23 15:05:10.537357 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:05:10.537318 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0bce6f_f74a_4e21_9921_a52bd3c30892.slice/crio-0240d372f0c36fea4aa155a6adefc193670202fee4ed0999f710116cb73856bf WatchSource:0}: Error finding container 0240d372f0c36fea4aa155a6adefc193670202fee4ed0999f710116cb73856bf: Status 404 returned error can't find the container with id 0240d372f0c36fea4aa155a6adefc193670202fee4ed0999f710116cb73856bf Jul 23 15:05:10.610022 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.609989 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765b7646dc-6bz67"] Jul 23 15:05:10.610583 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:05:10.610560 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4256b057_cb0d_4d5a_91c0_f08a8a400d03.slice/crio-8f507718b13e8090c9c264d278ea479535f1f08621178d10b793e5b2145d0574 WatchSource:0}: Error finding container 8f507718b13e8090c9c264d278ea479535f1f08621178d10b793e5b2145d0574: Status 404 returned error can't find the container with id 8f507718b13e8090c9c264d278ea479535f1f08621178d10b793e5b2145d0574 Jul 23 15:05:10.765223 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.765124 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-6bz67" event={"ID":"4256b057-cb0d-4d5a-91c0-f08a8a400d03","Type":"ContainerStarted","Data":"8f507718b13e8090c9c264d278ea479535f1f08621178d10b793e5b2145d0574"} Jul 23 15:05:10.766413 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:10.766391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859d6f66cd-bxz96" event={"ID":"cc0bce6f-f74a-4e21-9921-a52bd3c30892","Type":"ContainerStarted","Data":"0240d372f0c36fea4aa155a6adefc193670202fee4ed0999f710116cb73856bf"} Jul 23 15:05:13.788716 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:13.788678 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-6bz67" event={"ID":"4256b057-cb0d-4d5a-91c0-f08a8a400d03","Type":"ContainerStarted","Data":"0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93"} Jul 23 15:05:13.789175 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:13.788746 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:13.790129 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:13.790087 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859d6f66cd-bxz96" event={"ID":"cc0bce6f-f74a-4e21-9921-a52bd3c30892","Type":"ContainerStarted","Data":"b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417"} Jul 23 15:05:13.790251 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:13.790204 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:13.807651 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:13.807609 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-765b7646dc-6bz67" podStartSLOduration=1.074475007 podStartE2EDuration="3.807594094s" totalImagesPullingTime="2.733119087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:05:13.80633787 +0000 UTC m=+761.957787263" watchObservedRunningTime="2026-07-23 15:05:13.807594094 +0000 UTC m=+761.959043463" Jul 23 15:05:13.830616 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:13.830565 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859d6f66cd-bxz96" podStartSLOduration=2.014740453 podStartE2EDuration="4.830551178s" totalImagesPullingTime="2.815810725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:05:13.828631424 +0000 UTC m=+761.980080793" watchObservedRunningTime="2026-07-23 15:05:13.830551178 +0000 UTC m=+761.982000554" Jul 23 15:05:24.799523 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:24.799481 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:24.799942 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:24.799752 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:05:24.871073 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:24.871035 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859d6f66cd-bxz96"] Jul 23 15:05:24.871296 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:24.871210 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-859d6f66cd-bxz96" podUID="cc0bce6f-f74a-4e21-9921-a52bd3c30892" containerName="manager" containerID="cri-o://b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417" gracePeriod=30 Jul 23 15:05:25.126544 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.126517 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:25.253424 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.253392 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tnkc\" (UniqueName: \"kubernetes.io/projected/cc0bce6f-f74a-4e21-9921-a52bd3c30892-kube-api-access-4tnkc\") pod \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " Jul 23 15:05:25.253591 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.253466 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc0bce6f-f74a-4e21-9921-a52bd3c30892-webhook-cert\") pod \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\" (UID: \"cc0bce6f-f74a-4e21-9921-a52bd3c30892\") " Jul 23 15:05:25.255794 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.255763 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0bce6f-f74a-4e21-9921-a52bd3c30892-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cc0bce6f-f74a-4e21-9921-a52bd3c30892" (UID: "cc0bce6f-f74a-4e21-9921-a52bd3c30892"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:05:25.255934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.255822 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0bce6f-f74a-4e21-9921-a52bd3c30892-kube-api-access-4tnkc" (OuterVolumeSpecName: "kube-api-access-4tnkc") pod "cc0bce6f-f74a-4e21-9921-a52bd3c30892" (UID: "cc0bce6f-f74a-4e21-9921-a52bd3c30892"). InnerVolumeSpecName "kube-api-access-4tnkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:05:25.354311 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.354244 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc0bce6f-f74a-4e21-9921-a52bd3c30892-webhook-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:05:25.354311 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.354307 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tnkc\" (UniqueName: \"kubernetes.io/projected/cc0bce6f-f74a-4e21-9921-a52bd3c30892-kube-api-access-4tnkc\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:05:25.861783 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.861745 2582 generic.go:356] "Generic (PLEG): container finished" podID="cc0bce6f-f74a-4e21-9921-a52bd3c30892" containerID="b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417" exitCode=0 Jul 23 15:05:25.862283 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.861788 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859d6f66cd-bxz96" Jul 23 15:05:25.862283 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.861828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859d6f66cd-bxz96" event={"ID":"cc0bce6f-f74a-4e21-9921-a52bd3c30892","Type":"ContainerDied","Data":"b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417"} Jul 23 15:05:25.862283 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.861884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859d6f66cd-bxz96" event={"ID":"cc0bce6f-f74a-4e21-9921-a52bd3c30892","Type":"ContainerDied","Data":"0240d372f0c36fea4aa155a6adefc193670202fee4ed0999f710116cb73856bf"} Jul 23 15:05:25.862283 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.861912 2582 scope.go:117] "RemoveContainer" containerID="b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417" Jul 23 15:05:25.871767 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.871741 2582 scope.go:117] "RemoveContainer" containerID="b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417" Jul 23 15:05:25.872027 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:05:25.872010 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417\": container with ID starting with b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417 not found: ID does not exist" containerID="b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417" Jul 23 15:05:25.872082 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.872036 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417"} err="failed to get container status \"b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417\": rpc error: code = NotFound desc = could not find container \"b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417\": container with ID starting with b1ec4311898e1dd3e2c64dd5571ab1aded3b5402a1667cd1436e9fe629336417 not found: ID does not exist" Jul 23 15:05:25.885374 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.885351 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859d6f66cd-bxz96"] Jul 23 15:05:25.892298 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:25.892277 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-859d6f66cd-bxz96"] Jul 23 15:05:26.294473 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:26.294439 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0bce6f-f74a-4e21-9921-a52bd3c30892" path="/var/lib/kubelet/pods/cc0bce6f-f74a-4e21-9921-a52bd3c30892/volumes" Jul 23 15:05:46.180792 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.180746 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv"] Jul 23 15:05:46.181211 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.181186 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0bce6f-f74a-4e21-9921-a52bd3c30892" containerName="manager" Jul 23 15:05:46.181211 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.181199 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0bce6f-f74a-4e21-9921-a52bd3c30892" containerName="manager" Jul 23 15:05:46.181328 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.181282 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0bce6f-f74a-4e21-9921-a52bd3c30892" containerName="manager" Jul 23 15:05:46.189629 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.189608 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.193571 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.193548 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rdm8l\"" Jul 23 15:05:46.194180 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.194158 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 23 15:05:46.203710 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.203680 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv"] Jul 23 15:05:46.228398 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.228364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-openshift-service-ca\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.228576 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.228428 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt2wt\" (UniqueName: \"kubernetes.io/projected/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-kube-api-access-bt2wt\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.228576 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.228502 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-maas-api-tls\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.329470 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.329420 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt2wt\" (UniqueName: \"kubernetes.io/projected/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-kube-api-access-bt2wt\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.329683 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.329486 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-maas-api-tls\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.329683 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.329553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-openshift-service-ca\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.330243 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.330217 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-openshift-service-ca\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.332223 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.332203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-maas-api-tls\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.343177 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.343150 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt2wt\" (UniqueName: \"kubernetes.io/projected/4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5-kube-api-access-bt2wt\") pod \"maas-api-cb45bff9b-7tgjv\" (UID: \"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5\") " pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.500059 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.499974 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:46.661027 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.660992 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv"] Jul 23 15:05:46.662421 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:05:46.662385 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4acbb1ef_7753_46f8_b1d0_0f0aae5d4ca5.slice/crio-585d4e74e7b7e1d05cb862c41f33cd7de300abee55eb8cce750a5b3cc772b4eb WatchSource:0}: Error finding container 585d4e74e7b7e1d05cb862c41f33cd7de300abee55eb8cce750a5b3cc772b4eb: Status 404 returned error can't find the container with id 585d4e74e7b7e1d05cb862c41f33cd7de300abee55eb8cce750a5b3cc772b4eb Jul 23 15:05:46.719078 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.719049 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-c5n7x"] Jul 23 15:05:46.724124 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.724109 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.727898 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.727878 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w9g4s\"" Jul 23 15:05:46.728398 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.728358 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 15:05:46.733452 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.733431 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-c5n7x"] Jul 23 15:05:46.757021 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.756941 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-48vkr"] Jul 23 15:05:46.761561 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.761524 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.769881 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.769856 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-48vkr"] Jul 23 15:05:46.833888 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.833859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b3d89a9-99ab-4766-8a3b-35be00431657-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-c5n7x\" (UID: \"0b3d89a9-99ab-4766-8a3b-35be00431657\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.834071 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.833949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3dacc07-d2bf-4d9c-a10d-50ca81f4d603-plugins-config-volume\") pod \"payload-processing-5bf47cd988-48vkr\" (UID: \"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603\") " pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.834071 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.834055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7hw5\" (UniqueName: \"kubernetes.io/projected/0b3d89a9-99ab-4766-8a3b-35be00431657-kube-api-access-b7hw5\") pod \"payload-pre-processing-7677b774bb-c5n7x\" (UID: \"0b3d89a9-99ab-4766-8a3b-35be00431657\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.834154 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.834090 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k4pf\" (UniqueName: \"kubernetes.io/projected/e3dacc07-d2bf-4d9c-a10d-50ca81f4d603-kube-api-access-8k4pf\") pod \"payload-processing-5bf47cd988-48vkr\" (UID: \"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603\") " pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.935288 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.935224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k4pf\" (UniqueName: \"kubernetes.io/projected/e3dacc07-d2bf-4d9c-a10d-50ca81f4d603-kube-api-access-8k4pf\") pod \"payload-processing-5bf47cd988-48vkr\" (UID: \"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603\") " pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.935466 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.935306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b3d89a9-99ab-4766-8a3b-35be00431657-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-c5n7x\" (UID: \"0b3d89a9-99ab-4766-8a3b-35be00431657\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.935466 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.935347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3dacc07-d2bf-4d9c-a10d-50ca81f4d603-plugins-config-volume\") pod \"payload-processing-5bf47cd988-48vkr\" (UID: \"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603\") " pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.935466 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.935428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7hw5\" (UniqueName: \"kubernetes.io/projected/0b3d89a9-99ab-4766-8a3b-35be00431657-kube-api-access-b7hw5\") pod \"payload-pre-processing-7677b774bb-c5n7x\" (UID: \"0b3d89a9-99ab-4766-8a3b-35be00431657\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.936296 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.936245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3dacc07-d2bf-4d9c-a10d-50ca81f4d603-plugins-config-volume\") pod \"payload-processing-5bf47cd988-48vkr\" (UID: \"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603\") " pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.936296 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.936245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b3d89a9-99ab-4766-8a3b-35be00431657-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-c5n7x\" (UID: \"0b3d89a9-99ab-4766-8a3b-35be00431657\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.954366 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.954340 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7hw5\" (UniqueName: \"kubernetes.io/projected/0b3d89a9-99ab-4766-8a3b-35be00431657-kube-api-access-b7hw5\") pod \"payload-pre-processing-7677b774bb-c5n7x\" (UID: \"0b3d89a9-99ab-4766-8a3b-35be00431657\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:46.954542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.954427 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k4pf\" (UniqueName: \"kubernetes.io/projected/e3dacc07-d2bf-4d9c-a10d-50ca81f4d603-kube-api-access-8k4pf\") pod \"payload-processing-5bf47cd988-48vkr\" (UID: \"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603\") " pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:46.992867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:46.992830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" event={"ID":"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5","Type":"ContainerStarted","Data":"585d4e74e7b7e1d05cb862c41f33cd7de300abee55eb8cce750a5b3cc772b4eb"} Jul 23 15:05:47.037326 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:47.037223 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:47.074390 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:47.074313 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:47.223507 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:47.223468 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-c5n7x"] Jul 23 15:05:47.224768 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:05:47.224725 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3d89a9_99ab_4766_8a3b_35be00431657.slice/crio-0fc5e7bf1cc71a2e7d22ab81b87f08151897c79e9b871d1ed434b91f3e01171c WatchSource:0}: Error finding container 0fc5e7bf1cc71a2e7d22ab81b87f08151897c79e9b871d1ed434b91f3e01171c: Status 404 returned error can't find the container with id 0fc5e7bf1cc71a2e7d22ab81b87f08151897c79e9b871d1ed434b91f3e01171c Jul 23 15:05:47.269416 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:47.269384 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-48vkr"] Jul 23 15:05:47.271423 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:05:47.271398 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3dacc07_d2bf_4d9c_a10d_50ca81f4d603.slice/crio-da82cb3aec1b8443db5ed6737cf014f114816831d3e6355daba6a01349f9da26 WatchSource:0}: Error finding container da82cb3aec1b8443db5ed6737cf014f114816831d3e6355daba6a01349f9da26: Status 404 returned error can't find the container with id da82cb3aec1b8443db5ed6737cf014f114816831d3e6355daba6a01349f9da26 Jul 23 15:05:48.008358 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:48.008314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" event={"ID":"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603","Type":"ContainerStarted","Data":"da82cb3aec1b8443db5ed6737cf014f114816831d3e6355daba6a01349f9da26"} Jul 23 15:05:48.013243 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:48.011987 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" event={"ID":"0b3d89a9-99ab-4766-8a3b-35be00431657","Type":"ContainerStarted","Data":"0fc5e7bf1cc71a2e7d22ab81b87f08151897c79e9b871d1ed434b91f3e01171c"} Jul 23 15:05:49.021421 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:49.021382 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" event={"ID":"4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5","Type":"ContainerStarted","Data":"2176b37b88a17c1333a78fc5a2dcbf052de58b81614162ecd651334caa9a61b8"} Jul 23 15:05:49.021943 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:49.021539 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:05:49.043833 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:49.043787 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" podStartSLOduration=1.194392723 podStartE2EDuration="3.043772663s" totalImagesPullingTime="1.84937994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:05:49.0403163 +0000 UTC m=+797.191765684" watchObservedRunningTime="2026-07-23 15:05:49.043772663 +0000 UTC m=+797.195222031" Jul 23 15:05:50.029349 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:50.029311 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" event={"ID":"e3dacc07-d2bf-4d9c-a10d-50ca81f4d603","Type":"ContainerStarted","Data":"ff1926013c5637c262610fb16d8b44df646667e9071ab67726efd42af5b5d0ce"} Jul 23 15:05:50.029807 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:50.029714 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:05:50.030929 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:50.030905 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" event={"ID":"0b3d89a9-99ab-4766-8a3b-35be00431657","Type":"ContainerStarted","Data":"3996565c5ac252ab8a46150069a457beaf3a9f47c208306db11df4b4412278ac"} Jul 23 15:05:50.031101 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:50.031082 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:05:50.047273 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:50.047211 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" podStartSLOduration=1.980450338 podStartE2EDuration="4.047197434s" totalImagesPullingTime="2.066747096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:05:50.045522943 +0000 UTC m=+798.196972312" watchObservedRunningTime="2026-07-23 15:05:50.047197434 +0000 UTC m=+798.198646802" Jul 23 15:05:50.064171 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:50.064123 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" podStartSLOduration=2.235157432 podStartE2EDuration="4.064109353s" totalImagesPullingTime="1.828951921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:05:50.061806787 +0000 UTC m=+798.213256156" watchObservedRunningTime="2026-07-23 15:05:50.064109353 +0000 UTC m=+798.215558723" Jul 23 15:05:55.037011 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:05:55.036977 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-cb45bff9b-7tgjv" Jul 23 15:06:01.039448 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:01.039417 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-48vkr" Jul 23 15:06:01.039854 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:01.039494 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-c5n7x" Jul 23 15:06:18.733761 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.733714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt"] Jul 23 15:06:18.738627 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.738609 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.743188 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.743161 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 15:06:18.743477 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.743201 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nnqmb\"" Jul 23 15:06:18.743627 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.743210 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 23 15:06:18.744023 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.743303 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 15:06:18.746114 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.746086 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt"] Jul 23 15:06:18.747632 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.747499 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.747632 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.747575 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.747844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.747660 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.747844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.747689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fqnp\" (UniqueName: \"kubernetes.io/projected/64ce1d79-7bb3-4585-8795-d75b04d24909-kube-api-access-2fqnp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.747844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.747723 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64ce1d79-7bb3-4585-8795-d75b04d24909-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.747844 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.747825 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.848571 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.848769 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.848769 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.848769 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.848937 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fqnp\" (UniqueName: \"kubernetes.io/projected/64ce1d79-7bb3-4585-8795-d75b04d24909-kube-api-access-2fqnp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.848937 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848829 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64ce1d79-7bb3-4585-8795-d75b04d24909-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.849042 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.848990 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.849042 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.849019 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.849140 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.849075 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.851027 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.851007 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64ce1d79-7bb3-4585-8795-d75b04d24909-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.851351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.851336 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64ce1d79-7bb3-4585-8795-d75b04d24909-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:18.857757 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:18.857737 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fqnp\" (UniqueName: \"kubernetes.io/projected/64ce1d79-7bb3-4585-8795-d75b04d24909-kube-api-access-2fqnp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-29bdt\" (UID: \"64ce1d79-7bb3-4585-8795-d75b04d24909\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:19.050697 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:19.050605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:19.199498 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:19.199464 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt"] Jul 23 15:06:19.201379 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:06:19.201339 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ce1d79_7bb3_4585_8795_d75b04d24909.slice/crio-c662d281819afc63af4c841862dd1e615d23753df4fb0c4ba63737e64043b2de WatchSource:0}: Error finding container c662d281819afc63af4c841862dd1e615d23753df4fb0c4ba63737e64043b2de: Status 404 returned error can't find the container with id c662d281819afc63af4c841862dd1e615d23753df4fb0c4ba63737e64043b2de Jul 23 15:06:20.208983 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:20.208943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" event={"ID":"64ce1d79-7bb3-4585-8795-d75b04d24909","Type":"ContainerStarted","Data":"c662d281819afc63af4c841862dd1e615d23753df4fb0c4ba63737e64043b2de"} Jul 23 15:06:23.228674 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:23.228635 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" event={"ID":"64ce1d79-7bb3-4585-8795-d75b04d24909","Type":"ContainerStarted","Data":"2f61769fca8d58cf2e2be4e45f89c524ec765a2da5e18cf68270eea6ca258e8f"} Jul 23 15:06:23.247579 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:23.247527 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" podStartSLOduration=1.895750002 podStartE2EDuration="5.247512968s" totalImagesPullingTime="3.351762966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:06:23.246756783 +0000 UTC m=+831.398206151" watchObservedRunningTime="2026-07-23 15:06:23.247512968 +0000 UTC m=+831.398962365" Jul 23 15:06:29.051082 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:29.051043 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:29.051600 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:29.051115 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:29.063593 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:29.063568 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:29.275294 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:29.275248 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-29bdt" Jul 23 15:06:31.821748 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.821704 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2"] Jul 23 15:06:31.825456 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.825415 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.828369 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.828342 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 23 15:06:31.837279 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.837213 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2"] Jul 23 15:06:31.870099 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.870058 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.870332 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.870136 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9nd\" (UniqueName: \"kubernetes.io/projected/fd561a8a-4d3d-400b-b136-9db8355361af-kube-api-access-fm9nd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.870332 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.870174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.870332 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.870221 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.870332 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.870244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.870579 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.870382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd561a8a-4d3d-400b-b136-9db8355361af-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.971729 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.971673 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.971934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.971744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9nd\" (UniqueName: \"kubernetes.io/projected/fd561a8a-4d3d-400b-b136-9db8355361af-kube-api-access-fm9nd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.971934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.971779 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.971934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.971809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.971934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.971834 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.971934 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.971890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd561a8a-4d3d-400b-b136-9db8355361af-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.972534 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.972437 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.972929 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.972891 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.972929 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.972914 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.974914 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.974889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd561a8a-4d3d-400b-b136-9db8355361af-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.975456 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.975437 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd561a8a-4d3d-400b-b136-9db8355361af-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:31.980982 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:31.980963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9nd\" (UniqueName: \"kubernetes.io/projected/fd561a8a-4d3d-400b-b136-9db8355361af-kube-api-access-fm9nd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2\" (UID: \"fd561a8a-4d3d-400b-b136-9db8355361af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:32.140070 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:32.139970 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:32.295449 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:32.295421 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2"] Jul 23 15:06:32.296661 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:06:32.296635 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd561a8a_4d3d_400b_b136_9db8355361af.slice/crio-67152e61293513bee7eafc42eacc35d452b2e56335c64fbcc80a79183657437a WatchSource:0}: Error finding container 67152e61293513bee7eafc42eacc35d452b2e56335c64fbcc80a79183657437a: Status 404 returned error can't find the container with id 67152e61293513bee7eafc42eacc35d452b2e56335c64fbcc80a79183657437a Jul 23 15:06:33.289208 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:33.289171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" event={"ID":"fd561a8a-4d3d-400b-b136-9db8355361af","Type":"ContainerStarted","Data":"67152e61293513bee7eafc42eacc35d452b2e56335c64fbcc80a79183657437a"} Jul 23 15:06:34.297011 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:34.296974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" event={"ID":"fd561a8a-4d3d-400b-b136-9db8355361af","Type":"ContainerStarted","Data":"fb1999d680b9a2a7316a3182e423515324b6ae1fe6a1c2cfd5a8202e1262e07a"} Jul 23 15:06:34.316820 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:34.316771 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" podStartSLOduration=1.7561407 podStartE2EDuration="3.31675642s" totalImagesPullingTime="1.56061572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:06:34.315229276 +0000 UTC m=+842.466678646" watchObservedRunningTime="2026-07-23 15:06:34.31675642 +0000 UTC m=+842.468205793" Jul 23 15:06:42.140403 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:42.140364 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:42.140791 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:42.140431 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:42.153542 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:42.153513 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:06:42.354426 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:06:42.354396 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2" Jul 23 15:07:32.297830 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:32.297751 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:07:32.299212 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:32.299195 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:07:41.759538 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:41.759498 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cbdc6df48-4pnzl"] Jul 23 15:07:41.761135 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:41.759721 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" podUID="6ff28b92-0f8a-43ce-be6a-61bc069f21b3" containerName="authorino" containerID="cri-o://2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7" gracePeriod=30 Jul 23 15:07:42.031002 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.030977 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:07:42.118824 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.118794 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9crlf\" (UniqueName: \"kubernetes.io/projected/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-kube-api-access-9crlf\") pod \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " Jul 23 15:07:42.119003 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.118937 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-tls-cert\") pod \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\" (UID: \"6ff28b92-0f8a-43ce-be6a-61bc069f21b3\") " Jul 23 15:07:42.121104 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.121076 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-kube-api-access-9crlf" (OuterVolumeSpecName: "kube-api-access-9crlf") pod "6ff28b92-0f8a-43ce-be6a-61bc069f21b3" (UID: "6ff28b92-0f8a-43ce-be6a-61bc069f21b3"). InnerVolumeSpecName "kube-api-access-9crlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:07:42.132515 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.132488 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6ff28b92-0f8a-43ce-be6a-61bc069f21b3" (UID: "6ff28b92-0f8a-43ce-be6a-61bc069f21b3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:07:42.220373 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.220340 2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-tls-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:07:42.220373 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.220368 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9crlf\" (UniqueName: \"kubernetes.io/projected/6ff28b92-0f8a-43ce-be6a-61bc069f21b3-kube-api-access-9crlf\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:07:42.688449 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.688409 2582 generic.go:356] "Generic (PLEG): container finished" podID="6ff28b92-0f8a-43ce-be6a-61bc069f21b3" containerID="2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7" exitCode=0 Jul 23 15:07:42.688643 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.688453 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" Jul 23 15:07:42.688643 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.688496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" event={"ID":"6ff28b92-0f8a-43ce-be6a-61bc069f21b3","Type":"ContainerDied","Data":"2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7"} Jul 23 15:07:42.688643 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.688545 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbdc6df48-4pnzl" event={"ID":"6ff28b92-0f8a-43ce-be6a-61bc069f21b3","Type":"ContainerDied","Data":"2b6f4d49724c93efad279c896230c165c736e717ca730ad4a149888faf44417a"} Jul 23 15:07:42.688643 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.688563 2582 scope.go:117] "RemoveContainer" containerID="2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7" Jul 23 15:07:42.698409 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.698392 2582 scope.go:117] "RemoveContainer" containerID="2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7" Jul 23 15:07:42.698664 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:07:42.698641 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7\": container with ID starting with 2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7 not found: ID does not exist" containerID="2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7" Jul 23 15:07:42.698733 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.698676 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7"} err="failed to get container status \"2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7\": rpc error: code = NotFound desc = could not find container \"2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7\": container with ID starting with 2fa996a840721e38e59674c7cceb9b45b48a092c95bb8159fa84e11ff8f90cd7 not found: ID does not exist" Jul 23 15:07:42.708269 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.708229 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cbdc6df48-4pnzl"] Jul 23 15:07:42.711914 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:42.711893 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cbdc6df48-4pnzl"] Jul 23 15:07:44.294926 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:07:44.294895 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff28b92-0f8a-43ce-be6a-61bc069f21b3" path="/var/lib/kubelet/pods/6ff28b92-0f8a-43ce-be6a-61bc069f21b3/volumes" Jul 23 15:08:33.352485 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.352449 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765b7646dc-6bz67"] Jul 23 15:08:33.352983 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.352704 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-765b7646dc-6bz67" podUID="4256b057-cb0d-4d5a-91c0-f08a8a400d03" containerName="manager" containerID="cri-o://0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93" gracePeriod=30 Jul 23 15:08:33.394435 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:08:33.394397 2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4256b057_cb0d_4d5a_91c0_f08a8a400d03.slice/crio-0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93.scope\": RecentStats: unable to find data in memory cache]" Jul 23 15:08:33.619671 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.619643 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:08:33.797964 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.797929 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4256b057-cb0d-4d5a-91c0-f08a8a400d03-webhook-cert\") pod \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " Jul 23 15:08:33.798156 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.798053 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sfxc\" (UniqueName: \"kubernetes.io/projected/4256b057-cb0d-4d5a-91c0-f08a8a400d03-kube-api-access-6sfxc\") pod \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\" (UID: \"4256b057-cb0d-4d5a-91c0-f08a8a400d03\") " Jul 23 15:08:33.800111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.800084 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4256b057-cb0d-4d5a-91c0-f08a8a400d03-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4256b057-cb0d-4d5a-91c0-f08a8a400d03" (UID: "4256b057-cb0d-4d5a-91c0-f08a8a400d03"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:08:33.800206 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.800139 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4256b057-cb0d-4d5a-91c0-f08a8a400d03-kube-api-access-6sfxc" (OuterVolumeSpecName: "kube-api-access-6sfxc") pod "4256b057-cb0d-4d5a-91c0-f08a8a400d03" (UID: "4256b057-cb0d-4d5a-91c0-f08a8a400d03"). InnerVolumeSpecName "kube-api-access-6sfxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:08:33.899537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.899499 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sfxc\" (UniqueName: \"kubernetes.io/projected/4256b057-cb0d-4d5a-91c0-f08a8a400d03-kube-api-access-6sfxc\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:08:33.899537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.899538 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4256b057-cb0d-4d5a-91c0-f08a8a400d03-webhook-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:08:33.985828 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.985794 2582 generic.go:356] "Generic (PLEG): container finished" podID="4256b057-cb0d-4d5a-91c0-f08a8a400d03" containerID="0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93" exitCode=0 Jul 23 15:08:33.986013 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.985842 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-6bz67" Jul 23 15:08:33.986013 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.985883 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-6bz67" event={"ID":"4256b057-cb0d-4d5a-91c0-f08a8a400d03","Type":"ContainerDied","Data":"0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93"} Jul 23 15:08:33.986013 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.985939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-6bz67" event={"ID":"4256b057-cb0d-4d5a-91c0-f08a8a400d03","Type":"ContainerDied","Data":"8f507718b13e8090c9c264d278ea479535f1f08621178d10b793e5b2145d0574"} Jul 23 15:08:33.986013 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.985964 2582 scope.go:117] "RemoveContainer" containerID="0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93" Jul 23 15:08:33.996087 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.996070 2582 scope.go:117] "RemoveContainer" containerID="0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93" Jul 23 15:08:33.996377 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:08:33.996359 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93\": container with ID starting with 0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93 not found: ID does not exist" containerID="0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93" Jul 23 15:08:33.996428 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:33.996386 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93"} err="failed to get container status \"0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93\": rpc error: code = NotFound desc = could not find container \"0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93\": container with ID starting with 0aceb8c21f88b0828371f41ead98e35ed996e990c27cd3438fe9ed1d345b8f93 not found: ID does not exist" Jul 23 15:08:34.009961 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:34.009928 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765b7646dc-6bz67"] Jul 23 15:08:34.020001 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:34.019975 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-765b7646dc-6bz67"] Jul 23 15:08:34.301840 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:08:34.295994 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4256b057-cb0d-4d5a-91c0-f08a8a400d03" path="/var/lib/kubelet/pods/4256b057-cb0d-4d5a-91c0-f08a8a400d03/volumes" Jul 23 15:09:34.957620 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.957580 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-765b7646dc-h6mgn"] Jul 23 15:09:34.958065 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.958016 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff28b92-0f8a-43ce-be6a-61bc069f21b3" containerName="authorino" Jul 23 15:09:34.958065 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.958028 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff28b92-0f8a-43ce-be6a-61bc069f21b3" containerName="authorino" Jul 23 15:09:34.958065 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.958050 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4256b057-cb0d-4d5a-91c0-f08a8a400d03" containerName="manager" Jul 23 15:09:34.958065 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.958055 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4256b057-cb0d-4d5a-91c0-f08a8a400d03" containerName="manager" Jul 23 15:09:34.958221 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.958122 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff28b92-0f8a-43ce-be6a-61bc069f21b3" containerName="authorino" Jul 23 15:09:34.958221 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.958132 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4256b057-cb0d-4d5a-91c0-f08a8a400d03" containerName="manager" Jul 23 15:09:34.962315 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.962297 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:34.966678 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.966659 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j8sc5\"" Jul 23 15:09:34.966788 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.966661 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 15:09:34.966788 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.966661 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 15:09:34.971760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:34.971736 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765b7646dc-h6mgn"] Jul 23 15:09:35.060680 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.060647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-webhook-cert\") pod \"maas-controller-765b7646dc-h6mgn\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.060859 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.060772 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvg6\" (UniqueName: \"kubernetes.io/projected/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-kube-api-access-bgvg6\") pod \"maas-controller-765b7646dc-h6mgn\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.161351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.161312 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvg6\" (UniqueName: \"kubernetes.io/projected/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-kube-api-access-bgvg6\") pod \"maas-controller-765b7646dc-h6mgn\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.161525 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.161380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-webhook-cert\") pod \"maas-controller-765b7646dc-h6mgn\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.163909 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.163885 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-webhook-cert\") pod \"maas-controller-765b7646dc-h6mgn\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.170582 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.170553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvg6\" (UniqueName: \"kubernetes.io/projected/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-kube-api-access-bgvg6\") pod \"maas-controller-765b7646dc-h6mgn\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.274931 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.274843 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:35.417984 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.417954 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765b7646dc-h6mgn"] Jul 23 15:09:35.419983 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:09:35.419955 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f97a7d_83f2_4de3_98dc_e38aad4fe1fa.slice/crio-62a80d15c3c0a71c7d8efb283df2e2df4d643deeda74f1393e984726dc61e114 WatchSource:0}: Error finding container 62a80d15c3c0a71c7d8efb283df2e2df4d643deeda74f1393e984726dc61e114: Status 404 returned error can't find the container with id 62a80d15c3c0a71c7d8efb283df2e2df4d643deeda74f1393e984726dc61e114 Jul 23 15:09:35.421528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:35.421509 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:09:36.343537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:36.343439 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-h6mgn" event={"ID":"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa","Type":"ContainerStarted","Data":"2c59a98126e2f5b30e37903d59e1df8302ca74e5e68f4e520c937646f02d3cc5"} Jul 23 15:09:36.343537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:36.343495 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-h6mgn" event={"ID":"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa","Type":"ContainerStarted","Data":"62a80d15c3c0a71c7d8efb283df2e2df4d643deeda74f1393e984726dc61e114"} Jul 23 15:09:36.343945 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:36.343545 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:36.364168 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:36.364111 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-765b7646dc-h6mgn" podStartSLOduration=1.876450103 podStartE2EDuration="2.364096714s" totalImagesPullingTime="487.646611ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:09:36.362503492 +0000 UTC m=+1024.513952860" watchObservedRunningTime="2026-07-23 15:09:36.364096714 +0000 UTC m=+1024.515546083" Jul 23 15:09:47.352438 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:47.352404 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:52.119192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.119155 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765b7646dc-h6mgn"] Jul 23 15:09:52.119669 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.119371 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-765b7646dc-h6mgn" podUID="f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" containerName="manager" containerID="cri-o://2c59a98126e2f5b30e37903d59e1df8302ca74e5e68f4e520c937646f02d3cc5" gracePeriod=30 Jul 23 15:09:52.435498 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.435466 2582 generic.go:356] "Generic (PLEG): container finished" podID="f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" containerID="2c59a98126e2f5b30e37903d59e1df8302ca74e5e68f4e520c937646f02d3cc5" exitCode=0 Jul 23 15:09:52.435669 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.435550 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-h6mgn" event={"ID":"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa","Type":"ContainerDied","Data":"2c59a98126e2f5b30e37903d59e1df8302ca74e5e68f4e520c937646f02d3cc5"} Jul 23 15:09:52.482011 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.481983 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:52.504448 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.504417 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgvg6\" (UniqueName: \"kubernetes.io/projected/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-kube-api-access-bgvg6\") pod \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " Jul 23 15:09:52.504614 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.504488 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-webhook-cert\") pod \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\" (UID: \"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa\") " Jul 23 15:09:52.507574 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.507534 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-kube-api-access-bgvg6" (OuterVolumeSpecName: "kube-api-access-bgvg6") pod "f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" (UID: "f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa"). InnerVolumeSpecName "kube-api-access-bgvg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:09:52.507698 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.507580 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" (UID: "f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:09:52.605686 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.605653 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgvg6\" (UniqueName: \"kubernetes.io/projected/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-kube-api-access-bgvg6\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:09:52.605686 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:52.605682 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa-webhook-cert\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:09:53.442954 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:53.442913 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-h6mgn" Jul 23 15:09:53.443472 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:53.442959 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-h6mgn" event={"ID":"f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa","Type":"ContainerDied","Data":"62a80d15c3c0a71c7d8efb283df2e2df4d643deeda74f1393e984726dc61e114"} Jul 23 15:09:53.443472 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:53.443023 2582 scope.go:117] "RemoveContainer" containerID="2c59a98126e2f5b30e37903d59e1df8302ca74e5e68f4e520c937646f02d3cc5" Jul 23 15:09:53.476406 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:53.476371 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765b7646dc-h6mgn"] Jul 23 15:09:53.480978 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:53.480952 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-765b7646dc-h6mgn"] Jul 23 15:09:54.295018 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.294984 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" path="/var/lib/kubelet/pods/f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa/volumes" Jul 23 15:09:54.381373 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.381335 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-765b7646dc-qvtwr"] Jul 23 15:09:54.381822 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.381804 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" containerName="manager" Jul 23 15:09:54.381822 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.381820 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" containerName="manager" Jul 23 15:09:54.381991 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.381913 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f97a7d-83f2-4de3-98dc-e38aad4fe1fa" containerName="manager" Jul 23 15:09:54.386335 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.386316 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.389580 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.389560 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 15:09:54.390861 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.390840 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j8sc5\"" Jul 23 15:09:54.390861 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.390852 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 15:09:54.397155 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.397135 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765b7646dc-qvtwr"] Jul 23 15:09:54.421335 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.421300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddec958d-b408-4737-9759-12f5cfb6384a-webhook-cert\") pod \"maas-controller-765b7646dc-qvtwr\" (UID: \"ddec958d-b408-4737-9759-12f5cfb6384a\") " pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.421514 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.421406 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrblr\" (UniqueName: \"kubernetes.io/projected/ddec958d-b408-4737-9759-12f5cfb6384a-kube-api-access-xrblr\") pod \"maas-controller-765b7646dc-qvtwr\" (UID: \"ddec958d-b408-4737-9759-12f5cfb6384a\") " pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.522099 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.522001 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddec958d-b408-4737-9759-12f5cfb6384a-webhook-cert\") pod \"maas-controller-765b7646dc-qvtwr\" (UID: \"ddec958d-b408-4737-9759-12f5cfb6384a\") " pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.522099 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.522061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrblr\" (UniqueName: \"kubernetes.io/projected/ddec958d-b408-4737-9759-12f5cfb6384a-kube-api-access-xrblr\") pod \"maas-controller-765b7646dc-qvtwr\" (UID: \"ddec958d-b408-4737-9759-12f5cfb6384a\") " pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.524764 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.524742 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddec958d-b408-4737-9759-12f5cfb6384a-webhook-cert\") pod \"maas-controller-765b7646dc-qvtwr\" (UID: \"ddec958d-b408-4737-9759-12f5cfb6384a\") " pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.531151 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.531128 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrblr\" (UniqueName: \"kubernetes.io/projected/ddec958d-b408-4737-9759-12f5cfb6384a-kube-api-access-xrblr\") pod \"maas-controller-765b7646dc-qvtwr\" (UID: \"ddec958d-b408-4737-9759-12f5cfb6384a\") " pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.697919 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.697884 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:09:54.840796 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:54.840768 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765b7646dc-qvtwr"] Jul 23 15:09:54.842402 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:09:54.842375 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddec958d_b408_4737_9759_12f5cfb6384a.slice/crio-90a5520aaaaa3f402ec360ddbdf2d39872c762c5e7e1be1b265922611ab6167a WatchSource:0}: Error finding container 90a5520aaaaa3f402ec360ddbdf2d39872c762c5e7e1be1b265922611ab6167a: Status 404 returned error can't find the container with id 90a5520aaaaa3f402ec360ddbdf2d39872c762c5e7e1be1b265922611ab6167a Jul 23 15:09:55.457097 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:55.457059 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-qvtwr" event={"ID":"ddec958d-b408-4737-9759-12f5cfb6384a","Type":"ContainerStarted","Data":"90a5520aaaaa3f402ec360ddbdf2d39872c762c5e7e1be1b265922611ab6167a"} Jul 23 15:09:56.464586 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:56.464537 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765b7646dc-qvtwr" event={"ID":"ddec958d-b408-4737-9759-12f5cfb6384a","Type":"ContainerStarted","Data":"4280011c27ed51e04243fa183fb66d7ae9012f0bc054fd5dfbe6613799167a87"} Jul 23 15:09:56.464978 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:09:56.464736 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:10:07.473716 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:10:07.473678 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-765b7646dc-qvtwr" Jul 23 15:10:07.494471 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:10:07.494413 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-765b7646dc-qvtwr" podStartSLOduration=12.560087487 podStartE2EDuration="13.494396382s" totalImagesPullingTime="934.308895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:09:56.491358805 +0000 UTC m=+1044.642808175" watchObservedRunningTime="2026-07-23 15:10:07.494396382 +0000 UTC m=+1055.645845750" Jul 23 15:12:32.339148 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:12:32.339064 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:12:32.341203 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:12:32.341179 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:14:49.160559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:49.160518 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x"] Jul 23 15:14:49.161133 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:49.160717 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" podUID="667a7e00-a43f-4395-944a-30404fe7240d" containerName="manager" containerID="cri-o://a479833984a0cb7c03c8569d831216a9e7b95ab699c1cf146417012cfbaeceef" gracePeriod=10 Jul 23 15:14:52.205327 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.205294 2582 generic.go:356] "Generic (PLEG): container finished" podID="667a7e00-a43f-4395-944a-30404fe7240d" containerID="a479833984a0cb7c03c8569d831216a9e7b95ab699c1cf146417012cfbaeceef" exitCode=0 Jul 23 15:14:52.205708 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.205377 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" event={"ID":"667a7e00-a43f-4395-944a-30404fe7240d","Type":"ContainerDied","Data":"a479833984a0cb7c03c8569d831216a9e7b95ab699c1cf146417012cfbaeceef"} Jul 23 15:14:52.322618 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.322563 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:14:52.414716 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.414679 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/667a7e00-a43f-4395-944a-30404fe7240d-extensions-socket-volume\") pod \"667a7e00-a43f-4395-944a-30404fe7240d\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " Jul 23 15:14:52.414910 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.414740 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2bd7\" (UniqueName: \"kubernetes.io/projected/667a7e00-a43f-4395-944a-30404fe7240d-kube-api-access-z2bd7\") pod \"667a7e00-a43f-4395-944a-30404fe7240d\" (UID: \"667a7e00-a43f-4395-944a-30404fe7240d\") " Jul 23 15:14:52.415115 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.415091 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667a7e00-a43f-4395-944a-30404fe7240d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "667a7e00-a43f-4395-944a-30404fe7240d" (UID: "667a7e00-a43f-4395-944a-30404fe7240d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:14:52.416974 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.416954 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667a7e00-a43f-4395-944a-30404fe7240d-kube-api-access-z2bd7" (OuterVolumeSpecName: "kube-api-access-z2bd7") pod "667a7e00-a43f-4395-944a-30404fe7240d" (UID: "667a7e00-a43f-4395-944a-30404fe7240d"). InnerVolumeSpecName "kube-api-access-z2bd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:14:52.516551 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.516519 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2bd7\" (UniqueName: \"kubernetes.io/projected/667a7e00-a43f-4395-944a-30404fe7240d-kube-api-access-z2bd7\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:14:52.516551 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:52.516548 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/667a7e00-a43f-4395-944a-30404fe7240d-extensions-socket-volume\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:14:53.212500 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:53.212469 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" Jul 23 15:14:53.212955 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:53.212503 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x" event={"ID":"667a7e00-a43f-4395-944a-30404fe7240d","Type":"ContainerDied","Data":"6ba2ebad94dfff96a1f41d792e2d16bbf8f9a7ace2d90a24e052c4e52a86dbb2"} Jul 23 15:14:53.212955 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:53.212565 2582 scope.go:117] "RemoveContainer" containerID="a479833984a0cb7c03c8569d831216a9e7b95ab699c1cf146417012cfbaeceef" Jul 23 15:14:53.235893 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:53.235862 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x"] Jul 23 15:14:53.238770 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:53.238746 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbv4x"] Jul 23 15:14:54.295031 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:14:54.294998 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="667a7e00-a43f-4395-944a-30404fe7240d" path="/var/lib/kubelet/pods/667a7e00-a43f-4395-944a-30404fe7240d/volumes" Jul 23 15:15:00.149726 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.149689 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx"] Jul 23 15:15:00.150118 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.150108 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="667a7e00-a43f-4395-944a-30404fe7240d" containerName="manager" Jul 23 15:15:00.150161 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.150119 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="667a7e00-a43f-4395-944a-30404fe7240d" containerName="manager" Jul 23 15:15:00.150197 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.150188 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="667a7e00-a43f-4395-944a-30404fe7240d" containerName="manager" Jul 23 15:15:00.152614 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.152597 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" Jul 23 15:15:00.155589 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.155555 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:15:00.161226 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.161197 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx"] Jul 23 15:15:00.165882 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.165855 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" Jul 23 15:15:00.523177 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.523148 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx"] Jul 23 15:15:00.524273 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:15:00.524230 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1569c61_30c2_4b25_bb48_8c3cf64422ac.slice/crio-93f32d210da8a9a5d8d625ed8322d0f3ca7e4b82f08215f5c6a74c3f347b5d7b WatchSource:0}: Error finding container 93f32d210da8a9a5d8d625ed8322d0f3ca7e4b82f08215f5c6a74c3f347b5d7b: Status 404 returned error can't find the container with id 93f32d210da8a9a5d8d625ed8322d0f3ca7e4b82f08215f5c6a74c3f347b5d7b Jul 23 15:15:00.526166 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:00.526151 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:15:01.263051 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:01.263014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" event={"ID":"d1569c61-30c2-4b25-bb48-8c3cf64422ac","Type":"ContainerStarted","Data":"93f32d210da8a9a5d8d625ed8322d0f3ca7e4b82f08215f5c6a74c3f347b5d7b"} Jul 23 15:15:04.283149 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:04.283109 2582 generic.go:356] "Generic (PLEG): container finished" podID="d1569c61-30c2-4b25-bb48-8c3cf64422ac" containerID="d896caff8d3fb342601576c15b19be56f2e559e4a3c0c0b6adfc6dcad249a41c" exitCode=0 Jul 23 15:15:04.283636 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:04.283172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" event={"ID":"d1569c61-30c2-4b25-bb48-8c3cf64422ac","Type":"ContainerDied","Data":"d896caff8d3fb342601576c15b19be56f2e559e4a3c0c0b6adfc6dcad249a41c"} Jul 23 15:15:05.444387 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:05.444358 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" Jul 23 15:15:06.296756 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:06.296711 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" event={"ID":"d1569c61-30c2-4b25-bb48-8c3cf64422ac","Type":"ContainerDied","Data":"93f32d210da8a9a5d8d625ed8322d0f3ca7e4b82f08215f5c6a74c3f347b5d7b"} Jul 23 15:15:06.296756 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:06.296734 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746995-qvwjx" Jul 23 15:15:06.296972 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:06.296765 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93f32d210da8a9a5d8d625ed8322d0f3ca7e4b82f08215f5c6a74c3f347b5d7b" Jul 23 15:15:50.409039 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.409001 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7"] Jul 23 15:15:50.409473 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.409439 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1569c61-30c2-4b25-bb48-8c3cf64422ac" containerName="cleanup" Jul 23 15:15:50.409473 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.409451 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1569c61-30c2-4b25-bb48-8c3cf64422ac" containerName="cleanup" Jul 23 15:15:50.409546 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.409529 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1569c61-30c2-4b25-bb48-8c3cf64422ac" containerName="cleanup" Jul 23 15:15:50.413974 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.413948 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.416957 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.416938 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xsjqm\"" Jul 23 15:15:50.420700 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.420672 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9njbf\" (UniqueName: \"kubernetes.io/projected/b6379ab9-1845-4243-8cf6-3cd612f6df84-kube-api-access-9njbf\") pod \"kuadrant-operator-controller-manager-86878c769f-tfkc7\" (UID: \"b6379ab9-1845-4243-8cf6-3cd612f6df84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.420953 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.420928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6379ab9-1845-4243-8cf6-3cd612f6df84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tfkc7\" (UID: \"b6379ab9-1845-4243-8cf6-3cd612f6df84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.421918 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.421897 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7"] Jul 23 15:15:50.525763 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.525720 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6379ab9-1845-4243-8cf6-3cd612f6df84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tfkc7\" (UID: \"b6379ab9-1845-4243-8cf6-3cd612f6df84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.525964 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.525839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9njbf\" (UniqueName: \"kubernetes.io/projected/b6379ab9-1845-4243-8cf6-3cd612f6df84-kube-api-access-9njbf\") pod \"kuadrant-operator-controller-manager-86878c769f-tfkc7\" (UID: \"b6379ab9-1845-4243-8cf6-3cd612f6df84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.526104 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.526082 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6379ab9-1845-4243-8cf6-3cd612f6df84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tfkc7\" (UID: \"b6379ab9-1845-4243-8cf6-3cd612f6df84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.534592 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.534560 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9njbf\" (UniqueName: \"kubernetes.io/projected/b6379ab9-1845-4243-8cf6-3cd612f6df84-kube-api-access-9njbf\") pod \"kuadrant-operator-controller-manager-86878c769f-tfkc7\" (UID: \"b6379ab9-1845-4243-8cf6-3cd612f6df84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.726041 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.725946 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:50.869791 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:50.869765 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7"] Jul 23 15:15:50.872684 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:15:50.872657 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6379ab9_1845_4243_8cf6_3cd612f6df84.slice/crio-26bbf6939dc18ae35ec00167e31bd73e9aa8b318ddd8456ed3c3b6bee7f86fa5 WatchSource:0}: Error finding container 26bbf6939dc18ae35ec00167e31bd73e9aa8b318ddd8456ed3c3b6bee7f86fa5: Status 404 returned error can't find the container with id 26bbf6939dc18ae35ec00167e31bd73e9aa8b318ddd8456ed3c3b6bee7f86fa5 Jul 23 15:15:51.555914 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:51.555874 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" event={"ID":"b6379ab9-1845-4243-8cf6-3cd612f6df84","Type":"ContainerStarted","Data":"25be92ecf5efd1d7e60ac53a78da029128da68446d6171b14dd188b1a8907d9a"} Jul 23 15:15:51.556327 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:51.555937 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" event={"ID":"b6379ab9-1845-4243-8cf6-3cd612f6df84","Type":"ContainerStarted","Data":"26bbf6939dc18ae35ec00167e31bd73e9aa8b318ddd8456ed3c3b6bee7f86fa5"} Jul 23 15:15:51.556327 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:51.556059 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:15:51.576901 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:15:51.576853 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" podStartSLOduration=1.5768380610000001 podStartE2EDuration="1.576838061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:15:51.57456301 +0000 UTC m=+1399.726012400" watchObservedRunningTime="2026-07-23 15:15:51.576838061 +0000 UTC m=+1399.728287430" Jul 23 15:16:02.564232 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:16:02.564142 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tfkc7" Jul 23 15:17:32.383629 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:17:32.383596 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:17:32.385721 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:17:32.385700 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:22:32.421679 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:22:32.421647 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:22:32.424195 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:22:32.424175 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:23:45.891389 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:45.891245 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8"] Jul 23 15:23:45.894079 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:45.894058 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:45.897555 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:45.897241 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:23:45.901738 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:45.901713 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8"] Jul 23 15:23:46.014902 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:46.014862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/161c8f57-bb41-4471-90f5-379862c9a070-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8\" (UID: \"161c8f57-bb41-4471-90f5-379862c9a070\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:46.115418 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:46.115380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/161c8f57-bb41-4471-90f5-379862c9a070-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8\" (UID: \"161c8f57-bb41-4471-90f5-379862c9a070\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:46.116028 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:46.116005 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/161c8f57-bb41-4471-90f5-379862c9a070-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8\" (UID: \"161c8f57-bb41-4471-90f5-379862c9a070\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:46.208585 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:46.208545 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:46.358075 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:46.358035 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8"] Jul 23 15:23:46.360090 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:23:46.360060 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161c8f57_bb41_4471_90f5_379862c9a070.slice/crio-f89bd3d82c12937318d78867f438403cb2f56d511aa40bbb6cdd2881df413d16 WatchSource:0}: Error finding container f89bd3d82c12937318d78867f438403cb2f56d511aa40bbb6cdd2881df413d16: Status 404 returned error can't find the container with id f89bd3d82c12937318d78867f438403cb2f56d511aa40bbb6cdd2881df413d16 Jul 23 15:23:47.087764 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:23:47.087719 2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161c8f57_bb41_4471_90f5_379862c9a070.slice/crio-901ae1cd82d4e202661c40a5b8c3f14aad7aa8c134d25318128fad68c9fb27e9.scope\": RecentStats: unable to find data in memory cache]" Jul 23 15:23:47.346840 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:47.346745 2582 generic.go:356] "Generic (PLEG): container finished" podID="161c8f57-bb41-4471-90f5-379862c9a070" containerID="901ae1cd82d4e202661c40a5b8c3f14aad7aa8c134d25318128fad68c9fb27e9" exitCode=0 Jul 23 15:23:47.346989 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:47.346835 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" event={"ID":"161c8f57-bb41-4471-90f5-379862c9a070","Type":"ContainerDied","Data":"901ae1cd82d4e202661c40a5b8c3f14aad7aa8c134d25318128fad68c9fb27e9"} Jul 23 15:23:47.346989 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:47.346917 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" event={"ID":"161c8f57-bb41-4471-90f5-379862c9a070","Type":"ContainerStarted","Data":"f89bd3d82c12937318d78867f438403cb2f56d511aa40bbb6cdd2881df413d16"} Jul 23 15:23:48.491102 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:48.491077 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:48.638216 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:48.638122 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/161c8f57-bb41-4471-90f5-379862c9a070-maas-api-service-ca\") pod \"161c8f57-bb41-4471-90f5-379862c9a070\" (UID: \"161c8f57-bb41-4471-90f5-379862c9a070\") " Jul 23 15:23:48.638511 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:48.638488 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/161c8f57-bb41-4471-90f5-379862c9a070-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "161c8f57-bb41-4471-90f5-379862c9a070" (UID: "161c8f57-bb41-4471-90f5-379862c9a070"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:23:48.739691 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:48.739656 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/161c8f57-bb41-4471-90f5-379862c9a070-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:23:49.360020 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:49.359992 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" Jul 23 15:23:49.360192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:49.360032 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8" event={"ID":"161c8f57-bb41-4471-90f5-379862c9a070","Type":"ContainerDied","Data":"f89bd3d82c12937318d78867f438403cb2f56d511aa40bbb6cdd2881df413d16"} Jul 23 15:23:49.360192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:23:49.360082 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f89bd3d82c12937318d78867f438403cb2f56d511aa40bbb6cdd2881df413d16" Jul 23 15:24:00.961564 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:00.961473 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8"] Jul 23 15:24:00.964215 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:00.964185 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af2a71ee-xptk8"] Jul 23 15:24:02.296191 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:02.296155 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="161c8f57-bb41-4471-90f5-379862c9a070" path="/var/lib/kubelet/pods/161c8f57-bb41-4471-90f5-379862c9a070/volumes" Jul 23 15:24:29.663795 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.663758 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6"] Jul 23 15:24:29.664209 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.664196 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="161c8f57-bb41-4471-90f5-379862c9a070" containerName="revoke-keys" Jul 23 15:24:29.664269 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.664210 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="161c8f57-bb41-4471-90f5-379862c9a070" containerName="revoke-keys" Jul 23 15:24:29.664322 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.664302 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="161c8f57-bb41-4471-90f5-379862c9a070" containerName="revoke-keys" Jul 23 15:24:29.666760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.666732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:29.669588 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.669555 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:24:29.675010 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.674697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6"] Jul 23 15:24:29.832241 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.832203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc56db68-8b8b-49bf-8779-01c2310e99e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6\" (UID: \"dc56db68-8b8b-49bf-8779-01c2310e99e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:29.933126 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.933025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc56db68-8b8b-49bf-8779-01c2310e99e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6\" (UID: \"dc56db68-8b8b-49bf-8779-01c2310e99e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:29.933739 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.933711 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc56db68-8b8b-49bf-8779-01c2310e99e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6\" (UID: \"dc56db68-8b8b-49bf-8779-01c2310e99e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:29.979176 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:29.979138 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:30.133608 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:30.133579 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6"] Jul 23 15:24:30.135455 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:24:30.135426 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc56db68_8b8b_49bf_8779_01c2310e99e2.slice/crio-babef630541df24978caaae8cdcf10bd109d00aa7f85cdcf2d11aeb1f9e25fdd WatchSource:0}: Error finding container babef630541df24978caaae8cdcf10bd109d00aa7f85cdcf2d11aeb1f9e25fdd: Status 404 returned error can't find the container with id babef630541df24978caaae8cdcf10bd109d00aa7f85cdcf2d11aeb1f9e25fdd Jul 23 15:24:30.602937 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:30.602901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" event={"ID":"dc56db68-8b8b-49bf-8779-01c2310e99e2","Type":"ContainerStarted","Data":"829a97117a603a6048b38ea3a1014bc15033897abf5d4d27dc014013b8111884"} Jul 23 15:24:30.603121 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:30.602956 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" event={"ID":"dc56db68-8b8b-49bf-8779-01c2310e99e2","Type":"ContainerStarted","Data":"babef630541df24978caaae8cdcf10bd109d00aa7f85cdcf2d11aeb1f9e25fdd"} Jul 23 15:24:30.622202 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:30.622151 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" podStartSLOduration=1.622135431 podStartE2EDuration="1.622135431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:30.619857392 +0000 UTC m=+1918.771306762" watchObservedRunningTime="2026-07-23 15:24:30.622135431 +0000 UTC m=+1918.773584799" Jul 23 15:24:31.610409 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:31.610367 2582 generic.go:356] "Generic (PLEG): container finished" podID="dc56db68-8b8b-49bf-8779-01c2310e99e2" containerID="829a97117a603a6048b38ea3a1014bc15033897abf5d4d27dc014013b8111884" exitCode=0 Jul 23 15:24:31.610881 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:31.610458 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" event={"ID":"dc56db68-8b8b-49bf-8779-01c2310e99e2","Type":"ContainerDied","Data":"829a97117a603a6048b38ea3a1014bc15033897abf5d4d27dc014013b8111884"} Jul 23 15:24:32.755079 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:32.755054 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:32.857923 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:32.857885 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc56db68-8b8b-49bf-8779-01c2310e99e2-maas-api-service-ca\") pod \"dc56db68-8b8b-49bf-8779-01c2310e99e2\" (UID: \"dc56db68-8b8b-49bf-8779-01c2310e99e2\") " Jul 23 15:24:32.858235 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:32.858211 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc56db68-8b8b-49bf-8779-01c2310e99e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc56db68-8b8b-49bf-8779-01c2310e99e2" (UID: "dc56db68-8b8b-49bf-8779-01c2310e99e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:24:32.958676 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:32.958641 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc56db68-8b8b-49bf-8779-01c2310e99e2-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:24:33.623637 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:33.623611 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" Jul 23 15:24:33.623815 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:33.623637 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6" event={"ID":"dc56db68-8b8b-49bf-8779-01c2310e99e2","Type":"ContainerDied","Data":"babef630541df24978caaae8cdcf10bd109d00aa7f85cdcf2d11aeb1f9e25fdd"} Jul 23 15:24:33.623815 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:33.623681 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="babef630541df24978caaae8cdcf10bd109d00aa7f85cdcf2d11aeb1f9e25fdd" Jul 23 15:24:44.737903 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:44.737871 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6"] Jul 23 15:24:44.740302 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:44.740274 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e35a5c33-l8qp6"] Jul 23 15:24:46.295146 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:24:46.295109 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc56db68-8b8b-49bf-8779-01c2310e99e2" path="/var/lib/kubelet/pods/dc56db68-8b8b-49bf-8779-01c2310e99e2/volumes" Jul 23 15:25:14.645671 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.645630 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn"] Jul 23 15:25:14.646154 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.646076 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc56db68-8b8b-49bf-8779-01c2310e99e2" containerName="revoke-keys" Jul 23 15:25:14.646154 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.646088 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc56db68-8b8b-49bf-8779-01c2310e99e2" containerName="revoke-keys" Jul 23 15:25:14.646227 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.646175 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc56db68-8b8b-49bf-8779-01c2310e99e2" containerName="revoke-keys" Jul 23 15:25:14.649473 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.649456 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:14.652494 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.652469 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:25:14.657043 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.656638 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn"] Jul 23 15:25:14.717335 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.717300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc138c79-b6a8-4c87-8ad7-cef76b601d7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn\" (UID: \"dc138c79-b6a8-4c87-8ad7-cef76b601d7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:14.818116 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.818076 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc138c79-b6a8-4c87-8ad7-cef76b601d7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn\" (UID: \"dc138c79-b6a8-4c87-8ad7-cef76b601d7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:14.818736 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.818713 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc138c79-b6a8-4c87-8ad7-cef76b601d7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn\" (UID: \"dc138c79-b6a8-4c87-8ad7-cef76b601d7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:14.963363 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:14.963325 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:15.111429 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:15.111221 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn"] Jul 23 15:25:15.114028 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:25:15.113993 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc138c79_b6a8_4c87_8ad7_cef76b601d7a.slice/crio-f85146a8ba7e5725561cb6a88f550aa956b28e1ffb26b0982d316406c93c25e5 WatchSource:0}: Error finding container f85146a8ba7e5725561cb6a88f550aa956b28e1ffb26b0982d316406c93c25e5: Status 404 returned error can't find the container with id f85146a8ba7e5725561cb6a88f550aa956b28e1ffb26b0982d316406c93c25e5 Jul 23 15:25:15.888754 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:15.888722 2582 generic.go:356] "Generic (PLEG): container finished" podID="dc138c79-b6a8-4c87-8ad7-cef76b601d7a" containerID="1dd2b79485bccb49c1392042b3eda062a32bc4ebdc2322d645c4b85d9d90ae05" exitCode=0 Jul 23 15:25:15.889091 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:15.888812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" event={"ID":"dc138c79-b6a8-4c87-8ad7-cef76b601d7a","Type":"ContainerDied","Data":"1dd2b79485bccb49c1392042b3eda062a32bc4ebdc2322d645c4b85d9d90ae05"} Jul 23 15:25:15.889091 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:15.888858 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" event={"ID":"dc138c79-b6a8-4c87-8ad7-cef76b601d7a","Type":"ContainerStarted","Data":"f85146a8ba7e5725561cb6a88f550aa956b28e1ffb26b0982d316406c93c25e5"} Jul 23 15:25:17.042354 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.042327 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:17.137409 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.137379 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc138c79-b6a8-4c87-8ad7-cef76b601d7a-maas-api-service-ca\") pod \"dc138c79-b6a8-4c87-8ad7-cef76b601d7a\" (UID: \"dc138c79-b6a8-4c87-8ad7-cef76b601d7a\") " Jul 23 15:25:17.137796 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.137767 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc138c79-b6a8-4c87-8ad7-cef76b601d7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc138c79-b6a8-4c87-8ad7-cef76b601d7a" (UID: "dc138c79-b6a8-4c87-8ad7-cef76b601d7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:25:17.238691 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.238648 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc138c79-b6a8-4c87-8ad7-cef76b601d7a-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:25:17.902319 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.902291 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" Jul 23 15:25:17.902512 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.902313 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn" event={"ID":"dc138c79-b6a8-4c87-8ad7-cef76b601d7a","Type":"ContainerDied","Data":"f85146a8ba7e5725561cb6a88f550aa956b28e1ffb26b0982d316406c93c25e5"} Jul 23 15:25:17.902512 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:17.902362 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f85146a8ba7e5725561cb6a88f550aa956b28e1ffb26b0982d316406c93c25e5" Jul 23 15:25:29.709280 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:29.709208 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn"] Jul 23 15:25:29.711862 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:29.711839 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee7b3491-sjjdn"] Jul 23 15:25:30.295141 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:30.295106 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc138c79-b6a8-4c87-8ad7-cef76b601d7a" path="/var/lib/kubelet/pods/dc138c79-b6a8-4c87-8ad7-cef76b601d7a/volumes" Jul 23 15:25:58.808583 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.808546 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9"] Jul 23 15:25:58.809057 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.808968 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc138c79-b6a8-4c87-8ad7-cef76b601d7a" containerName="revoke-keys" Jul 23 15:25:58.809057 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.808979 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc138c79-b6a8-4c87-8ad7-cef76b601d7a" containerName="revoke-keys" Jul 23 15:25:58.809057 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.809040 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc138c79-b6a8-4c87-8ad7-cef76b601d7a" containerName="revoke-keys" Jul 23 15:25:58.812413 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.812396 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:25:58.815822 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.815797 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:25:58.819533 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.819510 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9"] Jul 23 15:25:58.925959 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:58.925929 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9\" (UID: \"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:25:59.026641 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:59.026604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9\" (UID: \"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:25:59.027237 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:59.027215 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9\" (UID: \"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:25:59.125589 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:59.125506 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:25:59.477763 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:25:59.477736 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9"] Jul 23 15:25:59.479219 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:25:59.479177 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a834c6_5f5c_44c4_a3a7_020dd3d00f4b.slice/crio-d8dfe4c82ec04b42775ed741931086fd6cb07dd0673a92a1e4de0186e6054ceb WatchSource:0}: Error finding container d8dfe4c82ec04b42775ed741931086fd6cb07dd0673a92a1e4de0186e6054ceb: Status 404 returned error can't find the container with id d8dfe4c82ec04b42775ed741931086fd6cb07dd0673a92a1e4de0186e6054ceb Jul 23 15:26:00.157025 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:00.156984 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" event={"ID":"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b","Type":"ContainerStarted","Data":"19f886c522f76bba983ee9a63962cacdb507e9495dcfaeefac2b29ced503a2bd"} Jul 23 15:26:00.157581 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:00.157092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" event={"ID":"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b","Type":"ContainerStarted","Data":"d8dfe4c82ec04b42775ed741931086fd6cb07dd0673a92a1e4de0186e6054ceb"} Jul 23 15:26:00.175041 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:00.174977 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" podStartSLOduration=2.174960584 podStartE2EDuration="2.174960584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:25:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:26:00.173319835 +0000 UTC m=+2008.324769202" watchObservedRunningTime="2026-07-23 15:26:00.174960584 +0000 UTC m=+2008.326409953" Jul 23 15:26:01.163887 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:01.163852 2582 generic.go:356] "Generic (PLEG): container finished" podID="38a834c6-5f5c-44c4-a3a7-020dd3d00f4b" containerID="19f886c522f76bba983ee9a63962cacdb507e9495dcfaeefac2b29ced503a2bd" exitCode=0 Jul 23 15:26:01.164321 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:01.163946 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" event={"ID":"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b","Type":"ContainerDied","Data":"19f886c522f76bba983ee9a63962cacdb507e9495dcfaeefac2b29ced503a2bd"} Jul 23 15:26:02.320392 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:02.320367 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:26:02.357594 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:02.357554 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b-maas-api-service-ca\") pod \"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b\" (UID: \"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b\") " Jul 23 15:26:02.358004 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:02.357977 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38a834c6-5f5c-44c4-a3a7-020dd3d00f4b" (UID: "38a834c6-5f5c-44c4-a3a7-020dd3d00f4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:26:02.459415 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:02.459319 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:26:03.177555 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:03.177526 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" Jul 23 15:26:03.177742 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:03.177570 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9" event={"ID":"38a834c6-5f5c-44c4-a3a7-020dd3d00f4b","Type":"ContainerDied","Data":"d8dfe4c82ec04b42775ed741931086fd6cb07dd0673a92a1e4de0186e6054ceb"} Jul 23 15:26:03.177742 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:03.177619 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8dfe4c82ec04b42775ed741931086fd6cb07dd0673a92a1e4de0186e6054ceb" Jul 23 15:26:13.887288 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:13.887233 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9"] Jul 23 15:26:13.890129 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:13.890105 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3e83224e-nvnz9"] Jul 23 15:26:14.295432 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:26:14.295401 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a834c6-5f5c-44c4-a3a7-020dd3d00f4b" path="/var/lib/kubelet/pods/38a834c6-5f5c-44c4-a3a7-020dd3d00f4b/volumes" Jul 23 15:27:32.469573 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:27:32.469543 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:27:32.470683 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:27:32.470662 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:28:07.940375 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.940336 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4"] Jul 23 15:28:07.940835 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.940773 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a834c6-5f5c-44c4-a3a7-020dd3d00f4b" containerName="revoke-keys" Jul 23 15:28:07.940835 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.940784 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a834c6-5f5c-44c4-a3a7-020dd3d00f4b" containerName="revoke-keys" Jul 23 15:28:07.940910 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.940857 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a834c6-5f5c-44c4-a3a7-020dd3d00f4b" containerName="revoke-keys" Jul 23 15:28:07.943982 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.943966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:07.946812 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.946786 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:28:07.950733 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:07.950697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4"] Jul 23 15:28:08.028865 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.028830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/367ef535-08b9-4e53-a475-7213051a5c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4\" (UID: \"367ef535-08b9-4e53-a475-7213051a5c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:08.129569 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.129533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/367ef535-08b9-4e53-a475-7213051a5c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4\" (UID: \"367ef535-08b9-4e53-a475-7213051a5c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:08.130231 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.130206 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/367ef535-08b9-4e53-a475-7213051a5c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4\" (UID: \"367ef535-08b9-4e53-a475-7213051a5c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:08.256111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.256025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:08.611871 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.611841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4"] Jul 23 15:28:08.613988 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:28:08.613960 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367ef535_08b9_4e53_a475_7213051a5c9c.slice/crio-99707fc453ff5e714fe1b21d95691a948b11fdf390f8504db03f4a84a95c6e15 WatchSource:0}: Error finding container 99707fc453ff5e714fe1b21d95691a948b11fdf390f8504db03f4a84a95c6e15: Status 404 returned error can't find the container with id 99707fc453ff5e714fe1b21d95691a948b11fdf390f8504db03f4a84a95c6e15 Jul 23 15:28:08.940187 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.940153 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" event={"ID":"367ef535-08b9-4e53-a475-7213051a5c9c","Type":"ContainerStarted","Data":"241a815123d9b243315a909ab8f84f41bcb512ad7128d80f62f6f44c2bfa838d"} Jul 23 15:28:08.940400 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.940215 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" event={"ID":"367ef535-08b9-4e53-a475-7213051a5c9c","Type":"ContainerStarted","Data":"99707fc453ff5e714fe1b21d95691a948b11fdf390f8504db03f4a84a95c6e15"} Jul 23 15:28:08.955477 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:08.955423 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" podStartSLOduration=1.9554070220000002 podStartE2EDuration="1.955407022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:28:08.954710462 +0000 UTC m=+2137.106159835" watchObservedRunningTime="2026-07-23 15:28:08.955407022 +0000 UTC m=+2137.106856391" Jul 23 15:28:09.947827 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:09.947791 2582 generic.go:356] "Generic (PLEG): container finished" podID="367ef535-08b9-4e53-a475-7213051a5c9c" containerID="241a815123d9b243315a909ab8f84f41bcb512ad7128d80f62f6f44c2bfa838d" exitCode=0 Jul 23 15:28:09.948279 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:09.947884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" event={"ID":"367ef535-08b9-4e53-a475-7213051a5c9c","Type":"ContainerDied","Data":"241a815123d9b243315a909ab8f84f41bcb512ad7128d80f62f6f44c2bfa838d"} Jul 23 15:28:11.100428 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.100400 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:11.158034 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.157996 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/367ef535-08b9-4e53-a475-7213051a5c9c-maas-api-service-ca\") pod \"367ef535-08b9-4e53-a475-7213051a5c9c\" (UID: \"367ef535-08b9-4e53-a475-7213051a5c9c\") " Jul 23 15:28:11.158399 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.158377 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/367ef535-08b9-4e53-a475-7213051a5c9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "367ef535-08b9-4e53-a475-7213051a5c9c" (UID: "367ef535-08b9-4e53-a475-7213051a5c9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:28:11.259232 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.259143 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/367ef535-08b9-4e53-a475-7213051a5c9c-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:28:11.961002 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.960972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" event={"ID":"367ef535-08b9-4e53-a475-7213051a5c9c","Type":"ContainerDied","Data":"99707fc453ff5e714fe1b21d95691a948b11fdf390f8504db03f4a84a95c6e15"} Jul 23 15:28:11.961178 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.961026 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99707fc453ff5e714fe1b21d95691a948b11fdf390f8504db03f4a84a95c6e15" Jul 23 15:28:11.961178 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:11.960983 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4" Jul 23 15:28:23.010288 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:23.010159 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4"] Jul 23 15:28:23.012872 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:23.012844 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7f6db9fe-97fn4"] Jul 23 15:28:24.296420 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:28:24.296383 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="367ef535-08b9-4e53-a475-7213051a5c9c" path="/var/lib/kubelet/pods/367ef535-08b9-4e53-a475-7213051a5c9c/volumes" Jul 23 15:29:30.796603 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.796565 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442"] Jul 23 15:29:30.797047 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.796993 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="367ef535-08b9-4e53-a475-7213051a5c9c" containerName="revoke-keys" Jul 23 15:29:30.797047 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.797003 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="367ef535-08b9-4e53-a475-7213051a5c9c" containerName="revoke-keys" Jul 23 15:29:30.797120 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.797069 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="367ef535-08b9-4e53-a475-7213051a5c9c" containerName="revoke-keys" Jul 23 15:29:30.800320 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.800301 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:30.803047 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.803027 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:29:30.807146 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.807124 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442"] Jul 23 15:29:30.916156 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:30.916117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d0d02a-95c6-4fe8-8afa-52166ee82124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442\" (UID: \"59d0d02a-95c6-4fe8-8afa-52166ee82124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:31.017215 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:31.017180 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d0d02a-95c6-4fe8-8afa-52166ee82124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442\" (UID: \"59d0d02a-95c6-4fe8-8afa-52166ee82124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:31.017917 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:31.017887 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d0d02a-95c6-4fe8-8afa-52166ee82124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442\" (UID: \"59d0d02a-95c6-4fe8-8afa-52166ee82124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:31.112775 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:31.112687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:31.262467 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:31.262440 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442"] Jul 23 15:29:31.263558 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:29:31.263534 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d0d02a_95c6_4fe8_8afa_52166ee82124.slice/crio-bc3138d279766746989b1e4437366b642e592b91b11565a7d8af4ca4ab9addc5 WatchSource:0}: Error finding container bc3138d279766746989b1e4437366b642e592b91b11565a7d8af4ca4ab9addc5: Status 404 returned error can't find the container with id bc3138d279766746989b1e4437366b642e592b91b11565a7d8af4ca4ab9addc5 Jul 23 15:29:31.470399 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:31.470357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" event={"ID":"59d0d02a-95c6-4fe8-8afa-52166ee82124","Type":"ContainerStarted","Data":"bc3138d279766746989b1e4437366b642e592b91b11565a7d8af4ca4ab9addc5"} Jul 23 15:29:32.480404 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:32.480363 2582 generic.go:356] "Generic (PLEG): container finished" podID="59d0d02a-95c6-4fe8-8afa-52166ee82124" containerID="b4fcd1213d0ced32c27985f21447cb9f1e510f12aa66f4ed7a882b63e464a693" exitCode=0 Jul 23 15:29:32.480787 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:32.480455 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" event={"ID":"59d0d02a-95c6-4fe8-8afa-52166ee82124","Type":"ContainerDied","Data":"b4fcd1213d0ced32c27985f21447cb9f1e510f12aa66f4ed7a882b63e464a693"} Jul 23 15:29:33.618614 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:33.618589 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:33.745350 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:33.745204 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d0d02a-95c6-4fe8-8afa-52166ee82124-maas-api-service-ca\") pod \"59d0d02a-95c6-4fe8-8afa-52166ee82124\" (UID: \"59d0d02a-95c6-4fe8-8afa-52166ee82124\") " Jul 23 15:29:33.745589 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:33.745567 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d0d02a-95c6-4fe8-8afa-52166ee82124-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59d0d02a-95c6-4fe8-8afa-52166ee82124" (UID: "59d0d02a-95c6-4fe8-8afa-52166ee82124"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:29:33.846936 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:33.846898 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d0d02a-95c6-4fe8-8afa-52166ee82124-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:29:34.494011 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:34.493971 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" event={"ID":"59d0d02a-95c6-4fe8-8afa-52166ee82124","Type":"ContainerDied","Data":"bc3138d279766746989b1e4437366b642e592b91b11565a7d8af4ca4ab9addc5"} Jul 23 15:29:34.494011 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:34.494008 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442" Jul 23 15:29:34.494233 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:34.494035 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc3138d279766746989b1e4437366b642e592b91b11565a7d8af4ca4ab9addc5" Jul 23 15:29:50.865077 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:50.865041 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442"] Jul 23 15:29:50.867928 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:50.867901 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2bd74f9-fb442"] Jul 23 15:29:52.295097 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:29:52.295064 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d0d02a-95c6-4fe8-8afa-52166ee82124" path="/var/lib/kubelet/pods/59d0d02a-95c6-4fe8-8afa-52166ee82124/volumes" Jul 23 15:30:00.137099 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.137058 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct"] Jul 23 15:30:00.137556 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.137541 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d0d02a-95c6-4fe8-8afa-52166ee82124" containerName="revoke-keys" Jul 23 15:30:00.137603 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.137559 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d0d02a-95c6-4fe8-8afa-52166ee82124" containerName="revoke-keys" Jul 23 15:30:00.137658 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.137648 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d0d02a-95c6-4fe8-8afa-52166ee82124" containerName="revoke-keys" Jul 23 15:30:00.141088 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.141068 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" Jul 23 15:30:00.144325 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.144297 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:30:00.147792 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.147754 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct"] Jul 23 15:30:00.154514 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.154486 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" Jul 23 15:30:00.313687 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.313662 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct"] Jul 23 15:30:00.315268 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:30:00.315222 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc7600f_7338_4205_b235_281658cf46b1.slice/crio-4183ebdded20fb7c38ecd39229843c8fd84797e166358d6cd9f20caddd2dd95b WatchSource:0}: Error finding container 4183ebdded20fb7c38ecd39229843c8fd84797e166358d6cd9f20caddd2dd95b: Status 404 returned error can't find the container with id 4183ebdded20fb7c38ecd39229843c8fd84797e166358d6cd9f20caddd2dd95b Jul 23 15:30:00.656673 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.656632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" event={"ID":"8cc7600f-7338-4205-b235-281658cf46b1","Type":"ContainerStarted","Data":"f905703321eef30c94005c32397199daba942b2104c5f52587a89faf0147a3fe"} Jul 23 15:30:00.656882 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.656736 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" event={"ID":"8cc7600f-7338-4205-b235-281658cf46b1","Type":"ContainerStarted","Data":"4183ebdded20fb7c38ecd39229843c8fd84797e166358d6cd9f20caddd2dd95b"} Jul 23 15:30:00.674130 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:00.674064 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" podStartSLOduration=0.674049082 podStartE2EDuration="674.049082ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:30:00.671351524 +0000 UTC m=+2248.822800893" watchObservedRunningTime="2026-07-23 15:30:00.674049082 +0000 UTC m=+2248.825498450" Jul 23 15:30:01.664957 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:01.664920 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cc7600f-7338-4205-b235-281658cf46b1" containerID="f905703321eef30c94005c32397199daba942b2104c5f52587a89faf0147a3fe" exitCode=0 Jul 23 15:30:01.665495 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:01.665003 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" event={"ID":"8cc7600f-7338-4205-b235-281658cf46b1","Type":"ContainerDied","Data":"f905703321eef30c94005c32397199daba942b2104c5f52587a89faf0147a3fe"} Jul 23 15:30:02.825524 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:02.825496 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" Jul 23 15:30:03.677881 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:03.677850 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" Jul 23 15:30:03.678089 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:03.677882 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747010-jrtct" event={"ID":"8cc7600f-7338-4205-b235-281658cf46b1","Type":"ContainerDied","Data":"4183ebdded20fb7c38ecd39229843c8fd84797e166358d6cd9f20caddd2dd95b"} Jul 23 15:30:03.678089 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:03.677927 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4183ebdded20fb7c38ecd39229843c8fd84797e166358d6cd9f20caddd2dd95b" Jul 23 15:30:32.915849 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:32.915807 2582 scope.go:117] "RemoveContainer" containerID="829a97117a603a6048b38ea3a1014bc15033897abf5d4d27dc014013b8111884" Jul 23 15:30:32.925192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:30:32.925169 2582 scope.go:117] "RemoveContainer" containerID="901ae1cd82d4e202661c40a5b8c3f14aad7aa8c134d25318128fad68c9fb27e9" Jul 23 15:31:32.966532 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:31:32.966437 2582 scope.go:117] "RemoveContainer" containerID="1dd2b79485bccb49c1392042b3eda062a32bc4ebdc2322d645c4b85d9d90ae05" Jul 23 15:32:32.515193 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:32:32.515161 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:32:32.517477 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:32:32.517456 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:32:32.991377 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:32:32.991350 2582 scope.go:117] "RemoveContainer" containerID="19f886c522f76bba983ee9a63962cacdb507e9495dcfaeefac2b29ced503a2bd" Jul 23 15:33:04.909982 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.909944 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz"] Jul 23 15:33:04.910456 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.910436 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc7600f-7338-4205-b235-281658cf46b1" containerName="cleanup" Jul 23 15:33:04.910456 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.910454 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc7600f-7338-4205-b235-281658cf46b1" containerName="cleanup" Jul 23 15:33:04.910543 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.910530 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc7600f-7338-4205-b235-281658cf46b1" containerName="cleanup" Jul 23 15:33:04.914102 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.914085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:04.917075 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.917054 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:33:04.920454 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:04.920431 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz"] Jul 23 15:33:05.011583 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.011542 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636bdb59-ff86-40bd-8c0b-223ff4bc3473-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz\" (UID: \"636bdb59-ff86-40bd-8c0b-223ff4bc3473\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:05.112912 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.112873 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636bdb59-ff86-40bd-8c0b-223ff4bc3473-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz\" (UID: \"636bdb59-ff86-40bd-8c0b-223ff4bc3473\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:05.113536 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.113514 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636bdb59-ff86-40bd-8c0b-223ff4bc3473-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz\" (UID: \"636bdb59-ff86-40bd-8c0b-223ff4bc3473\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:05.227571 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.227479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:05.373048 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.373002 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz"] Jul 23 15:33:05.374281 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:33:05.374238 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636bdb59_ff86_40bd_8c0b_223ff4bc3473.slice/crio-806882c9040e36dc25b34dde7bda28562e88f8650d946157cba6195ef5ad2dea WatchSource:0}: Error finding container 806882c9040e36dc25b34dde7bda28562e88f8650d946157cba6195ef5ad2dea: Status 404 returned error can't find the container with id 806882c9040e36dc25b34dde7bda28562e88f8650d946157cba6195ef5ad2dea Jul 23 15:33:05.785769 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.785716 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" event={"ID":"636bdb59-ff86-40bd-8c0b-223ff4bc3473","Type":"ContainerStarted","Data":"833739acae80ffce65d03de09a0176438119bd7b216989fa45f94634b856bca4"} Jul 23 15:33:05.785965 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.785796 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" event={"ID":"636bdb59-ff86-40bd-8c0b-223ff4bc3473","Type":"ContainerStarted","Data":"806882c9040e36dc25b34dde7bda28562e88f8650d946157cba6195ef5ad2dea"} Jul 23 15:33:05.803625 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:05.803578 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" podStartSLOduration=1.803559712 podStartE2EDuration="1.803559712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:33:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:33:05.801333092 +0000 UTC m=+2433.952782462" watchObservedRunningTime="2026-07-23 15:33:05.803559712 +0000 UTC m=+2433.955009081" Jul 23 15:33:06.792672 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:06.792634 2582 generic.go:356] "Generic (PLEG): container finished" podID="636bdb59-ff86-40bd-8c0b-223ff4bc3473" containerID="833739acae80ffce65d03de09a0176438119bd7b216989fa45f94634b856bca4" exitCode=0 Jul 23 15:33:06.793065 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:06.792721 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" event={"ID":"636bdb59-ff86-40bd-8c0b-223ff4bc3473","Type":"ContainerDied","Data":"833739acae80ffce65d03de09a0176438119bd7b216989fa45f94634b856bca4"} Jul 23 15:33:07.940751 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:07.940726 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:08.041829 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:08.041789 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636bdb59-ff86-40bd-8c0b-223ff4bc3473-maas-api-service-ca\") pod \"636bdb59-ff86-40bd-8c0b-223ff4bc3473\" (UID: \"636bdb59-ff86-40bd-8c0b-223ff4bc3473\") " Jul 23 15:33:08.042166 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:08.042144 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636bdb59-ff86-40bd-8c0b-223ff4bc3473-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "636bdb59-ff86-40bd-8c0b-223ff4bc3473" (UID: "636bdb59-ff86-40bd-8c0b-223ff4bc3473"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:33:08.143548 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:08.143462 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/636bdb59-ff86-40bd-8c0b-223ff4bc3473-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:33:08.805499 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:08.805467 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" Jul 23 15:33:08.805684 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:08.805508 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz" event={"ID":"636bdb59-ff86-40bd-8c0b-223ff4bc3473","Type":"ContainerDied","Data":"806882c9040e36dc25b34dde7bda28562e88f8650d946157cba6195ef5ad2dea"} Jul 23 15:33:08.805684 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:08.805561 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="806882c9040e36dc25b34dde7bda28562e88f8650d946157cba6195ef5ad2dea" Jul 23 15:33:19.993829 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:19.993793 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz"] Jul 23 15:33:19.996338 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:19.996316 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec2b36-jqppz"] Jul 23 15:33:20.294962 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:20.294866 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="636bdb59-ff86-40bd-8c0b-223ff4bc3473" path="/var/lib/kubelet/pods/636bdb59-ff86-40bd-8c0b-223ff4bc3473/volumes" Jul 23 15:33:26.959962 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.959921 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9"] Jul 23 15:33:26.960581 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.960561 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636bdb59-ff86-40bd-8c0b-223ff4bc3473" containerName="revoke-keys" Jul 23 15:33:26.960666 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.960584 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="636bdb59-ff86-40bd-8c0b-223ff4bc3473" containerName="revoke-keys" Jul 23 15:33:26.960729 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.960718 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="636bdb59-ff86-40bd-8c0b-223ff4bc3473" containerName="revoke-keys" Jul 23 15:33:26.964986 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.964963 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:26.968073 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.968047 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:33:26.970167 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:26.970137 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9"] Jul 23 15:33:27.119601 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.119559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38547eec-084f-4c81-b26c-6c1cf8fdc244-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9\" (UID: \"38547eec-084f-4c81-b26c-6c1cf8fdc244\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:27.220616 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.220499 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38547eec-084f-4c81-b26c-6c1cf8fdc244-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9\" (UID: \"38547eec-084f-4c81-b26c-6c1cf8fdc244\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:27.221140 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.221119 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38547eec-084f-4c81-b26c-6c1cf8fdc244-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9\" (UID: \"38547eec-084f-4c81-b26c-6c1cf8fdc244\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:27.279439 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.279399 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:27.432731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.431398 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9"] Jul 23 15:33:27.921892 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.921845 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" event={"ID":"38547eec-084f-4c81-b26c-6c1cf8fdc244","Type":"ContainerStarted","Data":"7f1584a80fad47d0a0832e27c30a886aab00a6f499f79efef0bedb04d917eb3c"} Jul 23 15:33:27.922070 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.921915 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" event={"ID":"38547eec-084f-4c81-b26c-6c1cf8fdc244","Type":"ContainerStarted","Data":"b817de001cd2bed3b748a73d54e6817196402a5e7faed42cbfba181a72182b77"} Jul 23 15:33:27.938990 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:27.938927 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" podStartSLOduration=1.938909921 podStartE2EDuration="1.938909921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:33:27.936934915 +0000 UTC m=+2456.088384321" watchObservedRunningTime="2026-07-23 15:33:27.938909921 +0000 UTC m=+2456.090359290" Jul 23 15:33:28.928821 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:28.928783 2582 generic.go:356] "Generic (PLEG): container finished" podID="38547eec-084f-4c81-b26c-6c1cf8fdc244" containerID="7f1584a80fad47d0a0832e27c30a886aab00a6f499f79efef0bedb04d917eb3c" exitCode=0 Jul 23 15:33:28.929246 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:28.928870 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" event={"ID":"38547eec-084f-4c81-b26c-6c1cf8fdc244","Type":"ContainerDied","Data":"7f1584a80fad47d0a0832e27c30a886aab00a6f499f79efef0bedb04d917eb3c"} Jul 23 15:33:30.093174 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.093145 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:30.146452 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.146417 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38547eec-084f-4c81-b26c-6c1cf8fdc244-maas-api-service-ca\") pod \"38547eec-084f-4c81-b26c-6c1cf8fdc244\" (UID: \"38547eec-084f-4c81-b26c-6c1cf8fdc244\") " Jul 23 15:33:30.146788 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.146762 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38547eec-084f-4c81-b26c-6c1cf8fdc244-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38547eec-084f-4c81-b26c-6c1cf8fdc244" (UID: "38547eec-084f-4c81-b26c-6c1cf8fdc244"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:33:30.247809 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.247719 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38547eec-084f-4c81-b26c-6c1cf8fdc244-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:33:30.941140 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.941106 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" Jul 23 15:33:30.941342 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.941141 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9" event={"ID":"38547eec-084f-4c81-b26c-6c1cf8fdc244","Type":"ContainerDied","Data":"b817de001cd2bed3b748a73d54e6817196402a5e7faed42cbfba181a72182b77"} Jul 23 15:33:30.941342 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:30.941192 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b817de001cd2bed3b748a73d54e6817196402a5e7faed42cbfba181a72182b77" Jul 23 15:33:42.026551 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:42.026517 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9"] Jul 23 15:33:42.029085 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:42.029063 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5c302e-dggg9"] Jul 23 15:33:42.295785 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:33:42.295705 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38547eec-084f-4c81-b26c-6c1cf8fdc244" path="/var/lib/kubelet/pods/38547eec-084f-4c81-b26c-6c1cf8fdc244/volumes" Jul 23 15:34:14.883939 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.883899 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg"] Jul 23 15:34:14.884437 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.884390 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38547eec-084f-4c81-b26c-6c1cf8fdc244" containerName="revoke-keys" Jul 23 15:34:14.884437 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.884404 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="38547eec-084f-4c81-b26c-6c1cf8fdc244" containerName="revoke-keys" Jul 23 15:34:14.884518 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.884493 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="38547eec-084f-4c81-b26c-6c1cf8fdc244" containerName="revoke-keys" Jul 23 15:34:14.887726 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.887706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:14.890568 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.890546 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:34:14.896131 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.896109 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg"] Jul 23 15:34:14.970899 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:14.970860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a00cf-c674-404f-b638-a563169cfede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg\" (UID: \"0d3a00cf-c674-404f-b638-a563169cfede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:15.072085 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:15.072045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a00cf-c674-404f-b638-a563169cfede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg\" (UID: \"0d3a00cf-c674-404f-b638-a563169cfede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:15.072944 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:15.072917 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a00cf-c674-404f-b638-a563169cfede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg\" (UID: \"0d3a00cf-c674-404f-b638-a563169cfede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:15.200744 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:15.200709 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:15.348093 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:15.348059 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg"] Jul 23 15:34:15.349950 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:34:15.349920 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3a00cf_c674_404f_b638_a563169cfede.slice/crio-f0497cb7acec8b2570f545d2678d1ee87fe53c254cd78e08b7ee4051cfa92056 WatchSource:0}: Error finding container f0497cb7acec8b2570f545d2678d1ee87fe53c254cd78e08b7ee4051cfa92056: Status 404 returned error can't find the container with id f0497cb7acec8b2570f545d2678d1ee87fe53c254cd78e08b7ee4051cfa92056 Jul 23 15:34:16.219031 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:16.218988 2582 generic.go:356] "Generic (PLEG): container finished" podID="0d3a00cf-c674-404f-b638-a563169cfede" containerID="18ddc902f2a28de22994b4cf4780fe13fd07651e4e179999a8300d1e7e89a3af" exitCode=0 Jul 23 15:34:16.219521 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:16.219031 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" event={"ID":"0d3a00cf-c674-404f-b638-a563169cfede","Type":"ContainerDied","Data":"18ddc902f2a28de22994b4cf4780fe13fd07651e4e179999a8300d1e7e89a3af"} Jul 23 15:34:16.219521 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:16.219086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" event={"ID":"0d3a00cf-c674-404f-b638-a563169cfede","Type":"ContainerStarted","Data":"f0497cb7acec8b2570f545d2678d1ee87fe53c254cd78e08b7ee4051cfa92056"} Jul 23 15:34:17.358592 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:17.358568 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:17.395578 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:17.395548 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a00cf-c674-404f-b638-a563169cfede-maas-api-service-ca\") pod \"0d3a00cf-c674-404f-b638-a563169cfede\" (UID: \"0d3a00cf-c674-404f-b638-a563169cfede\") " Jul 23 15:34:17.396043 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:17.396007 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3a00cf-c674-404f-b638-a563169cfede-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d3a00cf-c674-404f-b638-a563169cfede" (UID: "0d3a00cf-c674-404f-b638-a563169cfede"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:34:17.496698 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:17.496611 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a00cf-c674-404f-b638-a563169cfede-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:34:18.232082 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:18.232052 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" Jul 23 15:34:18.232292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:18.232054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg" event={"ID":"0d3a00cf-c674-404f-b638-a563169cfede","Type":"ContainerDied","Data":"f0497cb7acec8b2570f545d2678d1ee87fe53c254cd78e08b7ee4051cfa92056"} Jul 23 15:34:18.232292 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:18.232179 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0497cb7acec8b2570f545d2678d1ee87fe53c254cd78e08b7ee4051cfa92056" Jul 23 15:34:33.021362 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:33.021272 2582 scope.go:117] "RemoveContainer" containerID="241a815123d9b243315a909ab8f84f41bcb512ad7128d80f62f6f44c2bfa838d" Jul 23 15:34:34.950308 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:34.950270 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg"] Jul 23 15:34:34.952717 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:34.952692 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-fdfc03-rf5rg"] Jul 23 15:34:36.294274 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:36.294226 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3a00cf-c674-404f-b638-a563169cfede" path="/var/lib/kubelet/pods/0d3a00cf-c674-404f-b638-a563169cfede/volumes" Jul 23 15:34:41.691694 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.691660 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7"] Jul 23 15:34:41.692124 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.692110 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3a00cf-c674-404f-b638-a563169cfede" containerName="revoke-keys" Jul 23 15:34:41.692185 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.692128 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3a00cf-c674-404f-b638-a563169cfede" containerName="revoke-keys" Jul 23 15:34:41.692239 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.692224 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3a00cf-c674-404f-b638-a563169cfede" containerName="revoke-keys" Jul 23 15:34:41.695563 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.695542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:41.699370 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.699335 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:34:41.706173 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.706139 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7"] Jul 23 15:34:41.835126 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.835088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61953f55-2a4f-439c-8bf3-4a030371f851-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7\" (UID: \"61953f55-2a4f-439c-8bf3-4a030371f851\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:41.936453 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.936417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61953f55-2a4f-439c-8bf3-4a030371f851-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7\" (UID: \"61953f55-2a4f-439c-8bf3-4a030371f851\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:41.937130 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:41.937111 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61953f55-2a4f-439c-8bf3-4a030371f851-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7\" (UID: \"61953f55-2a4f-439c-8bf3-4a030371f851\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:42.016887 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:42.016790 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:42.176249 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:42.176212 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7"] Jul 23 15:34:42.177883 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:34:42.177851 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61953f55_2a4f_439c_8bf3_4a030371f851.slice/crio-7dd33ea8cd208cee4367e3ddce3e6103aa2d056c755871dcb6bcb418ee28f03e WatchSource:0}: Error finding container 7dd33ea8cd208cee4367e3ddce3e6103aa2d056c755871dcb6bcb418ee28f03e: Status 404 returned error can't find the container with id 7dd33ea8cd208cee4367e3ddce3e6103aa2d056c755871dcb6bcb418ee28f03e Jul 23 15:34:42.380559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:42.380485 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" event={"ID":"61953f55-2a4f-439c-8bf3-4a030371f851","Type":"ContainerStarted","Data":"7dd33ea8cd208cee4367e3ddce3e6103aa2d056c755871dcb6bcb418ee28f03e"} Jul 23 15:34:43.387644 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:43.387604 2582 generic.go:356] "Generic (PLEG): container finished" podID="61953f55-2a4f-439c-8bf3-4a030371f851" containerID="10985fed19bd30f71cd5a59bb0f3783eabc18626d039557e835b7caebebe30ee" exitCode=0 Jul 23 15:34:43.388037 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:43.387689 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" event={"ID":"61953f55-2a4f-439c-8bf3-4a030371f851","Type":"ContainerDied","Data":"10985fed19bd30f71cd5a59bb0f3783eabc18626d039557e835b7caebebe30ee"} Jul 23 15:34:44.530852 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:44.530824 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:44.662460 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:44.662430 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61953f55-2a4f-439c-8bf3-4a030371f851-maas-api-service-ca\") pod \"61953f55-2a4f-439c-8bf3-4a030371f851\" (UID: \"61953f55-2a4f-439c-8bf3-4a030371f851\") " Jul 23 15:34:44.662802 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:44.662780 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61953f55-2a4f-439c-8bf3-4a030371f851-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "61953f55-2a4f-439c-8bf3-4a030371f851" (UID: "61953f55-2a4f-439c-8bf3-4a030371f851"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:34:44.764048 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:44.764012 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61953f55-2a4f-439c-8bf3-4a030371f851-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:34:45.400701 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:45.400667 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" event={"ID":"61953f55-2a4f-439c-8bf3-4a030371f851","Type":"ContainerDied","Data":"7dd33ea8cd208cee4367e3ddce3e6103aa2d056c755871dcb6bcb418ee28f03e"} Jul 23 15:34:45.400701 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:45.400698 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7" Jul 23 15:34:45.400947 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:34:45.400717 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dd33ea8cd208cee4367e3ddce3e6103aa2d056c755871dcb6bcb418ee28f03e" Jul 23 15:35:01.755724 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:35:01.755687 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7"] Jul 23 15:35:01.758144 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:35:01.758120 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8bca8c-vnft7"] Jul 23 15:35:02.294831 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:35:02.294794 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61953f55-2a4f-439c-8bf3-4a030371f851" path="/var/lib/kubelet/pods/61953f55-2a4f-439c-8bf3-4a030371f851/volumes" Jul 23 15:35:33.043212 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:35:33.043175 2582 scope.go:117] "RemoveContainer" containerID="b4fcd1213d0ced32c27985f21447cb9f1e510f12aa66f4ed7a882b63e464a693" Jul 23 15:37:32.554678 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:37:32.554593 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:37:32.559907 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:37:32.559887 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:39:33.105311 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:39:33.105277 2582 scope.go:117] "RemoveContainer" containerID="833739acae80ffce65d03de09a0176438119bd7b216989fa45f94634b856bca4" Jul 23 15:39:33.115799 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:39:33.115761 2582 scope.go:117] "RemoveContainer" containerID="7f1584a80fad47d0a0832e27c30a886aab00a6f499f79efef0bedb04d917eb3c" Jul 23 15:40:33.142277 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:33.142231 2582 scope.go:117] "RemoveContainer" containerID="18ddc902f2a28de22994b4cf4780fe13fd07651e4e179999a8300d1e7e89a3af" Jul 23 15:40:58.652658 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.652619 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7950-140710425235120"] Jul 23 15:40:58.653141 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.653050 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61953f55-2a4f-439c-8bf3-4a030371f851" containerName="revoke-keys" Jul 23 15:40:58.653141 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.653061 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="61953f55-2a4f-439c-8bf3-4a030371f851" containerName="revoke-keys" Jul 23 15:40:58.653141 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.653122 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="61953f55-2a4f-439c-8bf3-4a030371f851" containerName="revoke-keys" Jul 23 15:40:58.656335 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.656319 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:40:58.659159 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.659136 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gwdt6\"" Jul 23 15:40:58.661354 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.661329 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425235120"] Jul 23 15:40:58.670424 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.670395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t669\" (UniqueName: \"kubernetes.io/projected/847ec864-ef79-449e-afaa-01dda1d21f40-kube-api-access-6t669\") pod \"test-curl-7950-140710425235120\" (UID: \"847ec864-ef79-449e-afaa-01dda1d21f40\") " pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:40:58.771375 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.771336 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t669\" (UniqueName: \"kubernetes.io/projected/847ec864-ef79-449e-afaa-01dda1d21f40-kube-api-access-6t669\") pod \"test-curl-7950-140710425235120\" (UID: \"847ec864-ef79-449e-afaa-01dda1d21f40\") " pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:40:58.780461 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.780423 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t669\" (UniqueName: \"kubernetes.io/projected/847ec864-ef79-449e-afaa-01dda1d21f40-kube-api-access-6t669\") pod \"test-curl-7950-140710425235120\" (UID: \"847ec864-ef79-449e-afaa-01dda1d21f40\") " pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:40:58.966605 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:58.966518 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:40:59.111951 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:59.111925 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425235120"] Jul 23 15:40:59.113958 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:40:59.113932 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod847ec864_ef79_449e_afaa_01dda1d21f40.slice/crio-a833c83a00897612567d2f33684b72eda0748e3509539cbca5cbd09a3f3290da WatchSource:0}: Error finding container a833c83a00897612567d2f33684b72eda0748e3509539cbca5cbd09a3f3290da: Status 404 returned error can't find the container with id a833c83a00897612567d2f33684b72eda0748e3509539cbca5cbd09a3f3290da Jul 23 15:40:59.115245 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:59.115228 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:40:59.682808 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:40:59.682756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425235120" event={"ID":"847ec864-ef79-449e-afaa-01dda1d21f40","Type":"ContainerStarted","Data":"a833c83a00897612567d2f33684b72eda0748e3509539cbca5cbd09a3f3290da"} Jul 23 15:41:00.691018 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:00.690980 2582 generic.go:356] "Generic (PLEG): container finished" podID="847ec864-ef79-449e-afaa-01dda1d21f40" containerID="cf25fe540e4723bd414a1882cd20ffb60270c6565d38931f593d07a5ad074503" exitCode=0 Jul 23 15:41:00.691018 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:00.691012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425235120" event={"ID":"847ec864-ef79-449e-afaa-01dda1d21f40","Type":"ContainerDied","Data":"cf25fe540e4723bd414a1882cd20ffb60270c6565d38931f593d07a5ad074503"} Jul 23 15:41:01.843577 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:01.843544 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:41:01.881119 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:01.881090 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7950-140710425235120_847ec864-ef79-449e-afaa-01dda1d21f40/test-curl-7950-140710425235120/0.log" Jul 23 15:41:01.903329 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:01.903287 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t669\" (UniqueName: \"kubernetes.io/projected/847ec864-ef79-449e-afaa-01dda1d21f40-kube-api-access-6t669\") pod \"847ec864-ef79-449e-afaa-01dda1d21f40\" (UID: \"847ec864-ef79-449e-afaa-01dda1d21f40\") " Jul 23 15:41:01.905683 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:01.905649 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847ec864-ef79-449e-afaa-01dda1d21f40-kube-api-access-6t669" (OuterVolumeSpecName: "kube-api-access-6t669") pod "847ec864-ef79-449e-afaa-01dda1d21f40" (UID: "847ec864-ef79-449e-afaa-01dda1d21f40"). InnerVolumeSpecName "kube-api-access-6t669". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:41:01.907201 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:01.907179 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7950-140710425235120"] Jul 23 15:41:01.910814 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:01.910791 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7950-140710425235120"] Jul 23 15:41:02.004856 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.004762 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t669\" (UniqueName: \"kubernetes.io/projected/847ec864-ef79-449e-afaa-01dda1d21f40-kube-api-access-6t669\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:02.051520 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.051479 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7950-140710425076400"] Jul 23 15:41:02.051944 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.051931 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847ec864-ef79-449e-afaa-01dda1d21f40" containerName="test-curl-7950-140710425235120" Jul 23 15:41:02.051990 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.051946 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="847ec864-ef79-449e-afaa-01dda1d21f40" containerName="test-curl-7950-140710425235120" Jul 23 15:41:02.052025 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.052017 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="847ec864-ef79-449e-afaa-01dda1d21f40" containerName="test-curl-7950-140710425235120" Jul 23 15:41:02.055146 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.055130 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:02.061225 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.061196 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425076400"] Jul 23 15:41:02.105550 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.105516 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv297\" (UniqueName: \"kubernetes.io/projected/9b903621-41cc-434a-b51c-82ae30252b4a-kube-api-access-vv297\") pod \"test-curl-7950-140710425076400\" (UID: \"9b903621-41cc-434a-b51c-82ae30252b4a\") " pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:02.207073 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.207032 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv297\" (UniqueName: \"kubernetes.io/projected/9b903621-41cc-434a-b51c-82ae30252b4a-kube-api-access-vv297\") pod \"test-curl-7950-140710425076400\" (UID: \"9b903621-41cc-434a-b51c-82ae30252b4a\") " pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:02.215101 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.215062 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv297\" (UniqueName: \"kubernetes.io/projected/9b903621-41cc-434a-b51c-82ae30252b4a-kube-api-access-vv297\") pod \"test-curl-7950-140710425076400\" (UID: \"9b903621-41cc-434a-b51c-82ae30252b4a\") " pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:02.294692 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.294592 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="847ec864-ef79-449e-afaa-01dda1d21f40" path="/var/lib/kubelet/pods/847ec864-ef79-449e-afaa-01dda1d21f40/volumes" Jul 23 15:41:02.366344 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.366308 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:02.511731 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.511700 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425076400"] Jul 23 15:41:02.512986 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:02.512960 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b903621_41cc_434a_b51c_82ae30252b4a.slice/crio-03c0e104eff27cfe84ab9b5b86337f906f32ec9937a11f764adbfdba64baa149 WatchSource:0}: Error finding container 03c0e104eff27cfe84ab9b5b86337f906f32ec9937a11f764adbfdba64baa149: Status 404 returned error can't find the container with id 03c0e104eff27cfe84ab9b5b86337f906f32ec9937a11f764adbfdba64baa149 Jul 23 15:41:02.704399 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.704360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425076400" event={"ID":"9b903621-41cc-434a-b51c-82ae30252b4a","Type":"ContainerStarted","Data":"03c0e104eff27cfe84ab9b5b86337f906f32ec9937a11f764adbfdba64baa149"} Jul 23 15:41:02.705568 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.705548 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425235120" Jul 23 15:41:02.705684 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:02.705576 2582 scope.go:117] "RemoveContainer" containerID="cf25fe540e4723bd414a1882cd20ffb60270c6565d38931f593d07a5ad074503" Jul 23 15:41:03.713076 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:03.713036 2582 generic.go:356] "Generic (PLEG): container finished" podID="9b903621-41cc-434a-b51c-82ae30252b4a" containerID="80ace9ee37ba3aa687f514a2f8ffb86cc56934ccf574930fa81496ca63e654ca" exitCode=0 Jul 23 15:41:03.713573 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:03.713128 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425076400" event={"ID":"9b903621-41cc-434a-b51c-82ae30252b4a","Type":"ContainerDied","Data":"80ace9ee37ba3aa687f514a2f8ffb86cc56934ccf574930fa81496ca63e654ca"} Jul 23 15:41:04.865830 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:04.865806 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:04.885817 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:04.885792 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7950-140710425076400_9b903621-41cc-434a-b51c-82ae30252b4a/test-curl-7950-140710425076400/0.log" Jul 23 15:41:04.908921 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:04.908884 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7950-140710425076400"] Jul 23 15:41:04.912476 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:04.912438 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7950-140710425076400"] Jul 23 15:41:04.935054 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:04.935028 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv297\" (UniqueName: \"kubernetes.io/projected/9b903621-41cc-434a-b51c-82ae30252b4a-kube-api-access-vv297\") pod \"9b903621-41cc-434a-b51c-82ae30252b4a\" (UID: \"9b903621-41cc-434a-b51c-82ae30252b4a\") " Jul 23 15:41:04.937250 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:04.937231 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b903621-41cc-434a-b51c-82ae30252b4a-kube-api-access-vv297" (OuterVolumeSpecName: "kube-api-access-vv297") pod "9b903621-41cc-434a-b51c-82ae30252b4a" (UID: "9b903621-41cc-434a-b51c-82ae30252b4a"). InnerVolumeSpecName "kube-api-access-vv297". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:41:05.035805 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.035727 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv297\" (UniqueName: \"kubernetes.io/projected/9b903621-41cc-434a-b51c-82ae30252b4a-kube-api-access-vv297\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:05.091303 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.091269 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:05.091699 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.091687 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b903621-41cc-434a-b51c-82ae30252b4a" containerName="test-curl-7950-140710425076400" Jul 23 15:41:05.091742 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.091701 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b903621-41cc-434a-b51c-82ae30252b4a" containerName="test-curl-7950-140710425076400" Jul 23 15:41:05.091787 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.091779 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b903621-41cc-434a-b51c-82ae30252b4a" containerName="test-curl-7950-140710425076400" Jul 23 15:41:05.094652 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.094637 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:05.103018 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.102305 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:05.136609 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.136580 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kcsj\" (UniqueName: \"kubernetes.io/projected/ba096d7b-ead0-4e62-abba-17942db751a3-kube-api-access-5kcsj\") pod \"test-curl-7950-140710425105440\" (UID: \"ba096d7b-ead0-4e62-abba-17942db751a3\") " pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:05.237888 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.237844 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kcsj\" (UniqueName: \"kubernetes.io/projected/ba096d7b-ead0-4e62-abba-17942db751a3-kube-api-access-5kcsj\") pod \"test-curl-7950-140710425105440\" (UID: \"ba096d7b-ead0-4e62-abba-17942db751a3\") " pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:05.246085 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.246056 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kcsj\" (UniqueName: \"kubernetes.io/projected/ba096d7b-ead0-4e62-abba-17942db751a3-kube-api-access-5kcsj\") pod \"test-curl-7950-140710425105440\" (UID: \"ba096d7b-ead0-4e62-abba-17942db751a3\") " pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:05.406019 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.405991 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:05.548910 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.548885 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:05.550920 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:05.550890 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba096d7b_ead0_4e62_abba_17942db751a3.slice/crio-b84222cda4b149c5557a971ac9b7bdaa0baf2d2c41fe09cc50a2d70f78263717 WatchSource:0}: Error finding container b84222cda4b149c5557a971ac9b7bdaa0baf2d2c41fe09cc50a2d70f78263717: Status 404 returned error can't find the container with id b84222cda4b149c5557a971ac9b7bdaa0baf2d2c41fe09cc50a2d70f78263717 Jul 23 15:41:05.727083 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.726993 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:05.727083 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.727000 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03c0e104eff27cfe84ab9b5b86337f906f32ec9937a11f764adbfdba64baa149" Jul 23 15:41:05.728282 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:05.728226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425105440" event={"ID":"ba096d7b-ead0-4e62-abba-17942db751a3","Type":"ContainerStarted","Data":"b84222cda4b149c5557a971ac9b7bdaa0baf2d2c41fe09cc50a2d70f78263717"} Jul 23 15:41:05.737773 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:41:05.737749 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7950-140710425076400\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="9b903621-41cc-434a-b51c-82ae30252b4a" pod="opendatahub/test-curl-7950-140710425076400" Jul 23 15:41:06.294198 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:06.294166 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b903621-41cc-434a-b51c-82ae30252b4a" path="/var/lib/kubelet/pods/9b903621-41cc-434a-b51c-82ae30252b4a/volumes" Jul 23 15:41:06.735748 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:06.735717 2582 generic.go:356] "Generic (PLEG): container finished" podID="ba096d7b-ead0-4e62-abba-17942db751a3" containerID="599717f68fff4a2da041bee60dafb9868893041203d3e2b24409d722b1b5e148" exitCode=0 Jul 23 15:41:06.735936 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:06.735808 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425105440" event={"ID":"ba096d7b-ead0-4e62-abba-17942db751a3","Type":"ContainerDied","Data":"599717f68fff4a2da041bee60dafb9868893041203d3e2b24409d722b1b5e148"} Jul 23 15:41:07.895403 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:07.895377 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:07.914244 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:07.914208 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7950-140710425105440_ba096d7b-ead0-4e62-abba-17942db751a3/test-curl-7950-140710425105440/0.log" Jul 23 15:41:07.939559 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:07.939512 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:07.942039 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:07.942005 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:07.963466 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:07.963437 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kcsj\" (UniqueName: \"kubernetes.io/projected/ba096d7b-ead0-4e62-abba-17942db751a3-kube-api-access-5kcsj\") pod \"ba096d7b-ead0-4e62-abba-17942db751a3\" (UID: \"ba096d7b-ead0-4e62-abba-17942db751a3\") " Jul 23 15:41:07.965701 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:07.965671 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba096d7b-ead0-4e62-abba-17942db751a3-kube-api-access-5kcsj" (OuterVolumeSpecName: "kube-api-access-5kcsj") pod "ba096d7b-ead0-4e62-abba-17942db751a3" (UID: "ba096d7b-ead0-4e62-abba-17942db751a3"). InnerVolumeSpecName "kube-api-access-5kcsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:41:08.064537 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.064438 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kcsj\" (UniqueName: \"kubernetes.io/projected/ba096d7b-ead0-4e62-abba-17942db751a3-kube-api-access-5kcsj\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:08.077192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.077161 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:08.077607 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.077594 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba096d7b-ead0-4e62-abba-17942db751a3" containerName="test-curl-7950-140710425105440" Jul 23 15:41:08.077672 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.077609 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba096d7b-ead0-4e62-abba-17942db751a3" containerName="test-curl-7950-140710425105440" Jul 23 15:41:08.077713 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.077690 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba096d7b-ead0-4e62-abba-17942db751a3" containerName="test-curl-7950-140710425105440" Jul 23 15:41:08.080806 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.080790 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:08.087408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.087381 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:08.165792 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.165756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k5jp\" (UniqueName: \"kubernetes.io/projected/4ef14329-f987-4675-b9ac-09c65ffa9d8c-kube-api-access-5k5jp\") pod \"test-curl-7950-140710425105440\" (UID: \"4ef14329-f987-4675-b9ac-09c65ffa9d8c\") " pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:08.267323 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.267283 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k5jp\" (UniqueName: \"kubernetes.io/projected/4ef14329-f987-4675-b9ac-09c65ffa9d8c-kube-api-access-5k5jp\") pod \"test-curl-7950-140710425105440\" (UID: \"4ef14329-f987-4675-b9ac-09c65ffa9d8c\") " pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:08.275672 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.275636 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k5jp\" (UniqueName: \"kubernetes.io/projected/4ef14329-f987-4675-b9ac-09c65ffa9d8c-kube-api-access-5k5jp\") pod \"test-curl-7950-140710425105440\" (UID: \"4ef14329-f987-4675-b9ac-09c65ffa9d8c\") " pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:08.294990 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.294956 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba096d7b-ead0-4e62-abba-17942db751a3" path="/var/lib/kubelet/pods/ba096d7b-ead0-4e62-abba-17942db751a3/volumes" Jul 23 15:41:08.392417 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.392330 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:08.538506 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.538477 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:08.540204 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:08.540182 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ef14329_f987_4675_b9ac_09c65ffa9d8c.slice/crio-67e6a7f5096422690e2d5451b33bbcfda63bae68eab80935f48d1ec7bed140e1 WatchSource:0}: Error finding container 67e6a7f5096422690e2d5451b33bbcfda63bae68eab80935f48d1ec7bed140e1: Status 404 returned error can't find the container with id 67e6a7f5096422690e2d5451b33bbcfda63bae68eab80935f48d1ec7bed140e1 Jul 23 15:41:08.747843 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.747808 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425105440" event={"ID":"4ef14329-f987-4675-b9ac-09c65ffa9d8c","Type":"ContainerStarted","Data":"67e6a7f5096422690e2d5451b33bbcfda63bae68eab80935f48d1ec7bed140e1"} Jul 23 15:41:08.749060 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.749034 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:08.749187 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:08.749040 2582 scope.go:117] "RemoveContainer" containerID="599717f68fff4a2da041bee60dafb9868893041203d3e2b24409d722b1b5e148" Jul 23 15:41:09.756697 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:09.756664 2582 generic.go:356] "Generic (PLEG): container finished" podID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" containerID="f8b384d615d9bbb5e09f2d02d8d3cb3ec6ad897286ec42f5e3151308961f8ace" exitCode=0 Jul 23 15:41:09.757156 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:09.756752 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425105440" event={"ID":"4ef14329-f987-4675-b9ac-09c65ffa9d8c","Type":"ContainerDied","Data":"f8b384d615d9bbb5e09f2d02d8d3cb3ec6ad897286ec42f5e3151308961f8ace"} Jul 23 15:41:10.928890 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:10.928855 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:10.949941 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:10.949909 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7950-140710425105440_4ef14329-f987-4675-b9ac-09c65ffa9d8c/test-curl-7950-140710425105440/0.log" Jul 23 15:41:10.979539 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:10.979488 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:10.984218 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:10.984189 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7950-140710425105440"] Jul 23 15:41:10.999155 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:10.999130 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k5jp\" (UniqueName: \"kubernetes.io/projected/4ef14329-f987-4675-b9ac-09c65ffa9d8c-kube-api-access-5k5jp\") pod \"4ef14329-f987-4675-b9ac-09c65ffa9d8c\" (UID: \"4ef14329-f987-4675-b9ac-09c65ffa9d8c\") " Jul 23 15:41:11.001386 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.001361 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef14329-f987-4675-b9ac-09c65ffa9d8c-kube-api-access-5k5jp" (OuterVolumeSpecName: "kube-api-access-5k5jp") pod "4ef14329-f987-4675-b9ac-09c65ffa9d8c" (UID: "4ef14329-f987-4675-b9ac-09c65ffa9d8c"). InnerVolumeSpecName "kube-api-access-5k5jp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:41:11.100952 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.100835 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k5jp\" (UniqueName: \"kubernetes.io/projected/4ef14329-f987-4675-b9ac-09c65ffa9d8c-kube-api-access-5k5jp\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:11.125524 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.125487 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:11.125990 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.125974 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" containerName="test-curl-7950-140710425105440" Jul 23 15:41:11.126037 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.125992 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" containerName="test-curl-7950-140710425105440" Jul 23 15:41:11.126073 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.126063 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" containerName="test-curl-7950-140710425105440" Jul 23 15:41:11.129056 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.129040 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:11.135111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.135079 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:11.202145 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.202108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgvk\" (UniqueName: \"kubernetes.io/projected/7aad657c-95eb-4ac3-b959-d9b059bb0e86-kube-api-access-5tgvk\") pod \"test-curl-7950-140710425106448\" (UID: \"7aad657c-95eb-4ac3-b959-d9b059bb0e86\") " pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:11.303351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.303308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgvk\" (UniqueName: \"kubernetes.io/projected/7aad657c-95eb-4ac3-b959-d9b059bb0e86-kube-api-access-5tgvk\") pod \"test-curl-7950-140710425106448\" (UID: \"7aad657c-95eb-4ac3-b959-d9b059bb0e86\") " pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:11.311313 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.311288 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgvk\" (UniqueName: \"kubernetes.io/projected/7aad657c-95eb-4ac3-b959-d9b059bb0e86-kube-api-access-5tgvk\") pod \"test-curl-7950-140710425106448\" (UID: \"7aad657c-95eb-4ac3-b959-d9b059bb0e86\") " pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:11.440825 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.440790 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:11.582918 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.582881 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:11.583100 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:11.583050 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7aad657c_95eb_4ac3_b959_d9b059bb0e86.slice/crio-949fce1cbfa6da412882ef6449edd56c14559eaa0732466fe470e27e2c659a04 WatchSource:0}: Error finding container 949fce1cbfa6da412882ef6449edd56c14559eaa0732466fe470e27e2c659a04: Status 404 returned error can't find the container with id 949fce1cbfa6da412882ef6449edd56c14559eaa0732466fe470e27e2c659a04 Jul 23 15:41:11.770943 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.770849 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425106448" event={"ID":"7aad657c-95eb-4ac3-b959-d9b059bb0e86","Type":"ContainerStarted","Data":"949fce1cbfa6da412882ef6449edd56c14559eaa0732466fe470e27e2c659a04"} Jul 23 15:41:11.772111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.772081 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67e6a7f5096422690e2d5451b33bbcfda63bae68eab80935f48d1ec7bed140e1" Jul 23 15:41:11.772111 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:11.772099 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:11.782835 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:41:11.782801 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7950-140710425105440\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:12.294567 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:41:12.294522 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7950-140710425105440\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" pod="opendatahub/test-curl-7950-140710425105440" Jul 23 15:41:12.297206 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:12.297180 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ef14329-f987-4675-b9ac-09c65ffa9d8c" path="/var/lib/kubelet/pods/4ef14329-f987-4675-b9ac-09c65ffa9d8c/volumes" Jul 23 15:41:12.779664 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:12.779626 2582 generic.go:356] "Generic (PLEG): container finished" podID="7aad657c-95eb-4ac3-b959-d9b059bb0e86" containerID="1843b22765ab9fdf647768ea4e61ddfb0aef3b7d5ddca431f8b95fe36664a46e" exitCode=0 Jul 23 15:41:12.779820 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:12.779705 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425106448" event={"ID":"7aad657c-95eb-4ac3-b959-d9b059bb0e86","Type":"ContainerDied","Data":"1843b22765ab9fdf647768ea4e61ddfb0aef3b7d5ddca431f8b95fe36664a46e"} Jul 23 15:41:13.945429 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:13.945402 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:13.964558 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:13.964523 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7950-140710425106448_7aad657c-95eb-4ac3-b959-d9b059bb0e86/test-curl-7950-140710425106448/0.log" Jul 23 15:41:13.990648 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:13.990620 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:13.993386 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:13.993363 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:14.027501 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.027468 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tgvk\" (UniqueName: \"kubernetes.io/projected/7aad657c-95eb-4ac3-b959-d9b059bb0e86-kube-api-access-5tgvk\") pod \"7aad657c-95eb-4ac3-b959-d9b059bb0e86\" (UID: \"7aad657c-95eb-4ac3-b959-d9b059bb0e86\") " Jul 23 15:41:14.029853 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.029820 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aad657c-95eb-4ac3-b959-d9b059bb0e86-kube-api-access-5tgvk" (OuterVolumeSpecName: "kube-api-access-5tgvk") pod "7aad657c-95eb-4ac3-b959-d9b059bb0e86" (UID: "7aad657c-95eb-4ac3-b959-d9b059bb0e86"). InnerVolumeSpecName "kube-api-access-5tgvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:41:14.128909 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.128810 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tgvk\" (UniqueName: \"kubernetes.io/projected/7aad657c-95eb-4ac3-b959-d9b059bb0e86-kube-api-access-5tgvk\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:14.132237 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.132212 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:14.132714 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.132697 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aad657c-95eb-4ac3-b959-d9b059bb0e86" containerName="test-curl-7950-140710425106448" Jul 23 15:41:14.132761 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.132716 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aad657c-95eb-4ac3-b959-d9b059bb0e86" containerName="test-curl-7950-140710425106448" Jul 23 15:41:14.132808 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.132799 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aad657c-95eb-4ac3-b959-d9b059bb0e86" containerName="test-curl-7950-140710425106448" Jul 23 15:41:14.135837 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.135822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:14.151677 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.151589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:14.229512 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.229467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9trt\" (UniqueName: \"kubernetes.io/projected/239314fd-f44b-4f6e-abe4-7e64eb3a3176-kube-api-access-v9trt\") pod \"test-curl-7950-140710425106448\" (UID: \"239314fd-f44b-4f6e-abe4-7e64eb3a3176\") " pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:14.295977 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.295941 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aad657c-95eb-4ac3-b959-d9b059bb0e86" path="/var/lib/kubelet/pods/7aad657c-95eb-4ac3-b959-d9b059bb0e86/volumes" Jul 23 15:41:14.330657 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.330626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9trt\" (UniqueName: \"kubernetes.io/projected/239314fd-f44b-4f6e-abe4-7e64eb3a3176-kube-api-access-v9trt\") pod \"test-curl-7950-140710425106448\" (UID: \"239314fd-f44b-4f6e-abe4-7e64eb3a3176\") " pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:14.338982 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.338952 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9trt\" (UniqueName: \"kubernetes.io/projected/239314fd-f44b-4f6e-abe4-7e64eb3a3176-kube-api-access-v9trt\") pod \"test-curl-7950-140710425106448\" (UID: \"239314fd-f44b-4f6e-abe4-7e64eb3a3176\") " pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:14.450596 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.450561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:14.593115 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.593090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:14.594775 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:14.594746 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod239314fd_f44b_4f6e_abe4_7e64eb3a3176.slice/crio-0f03b69a2f085279b916f93be4d6cd42e75d3b038d8b9d9cee60c768efe3a1c2 WatchSource:0}: Error finding container 0f03b69a2f085279b916f93be4d6cd42e75d3b038d8b9d9cee60c768efe3a1c2: Status 404 returned error can't find the container with id 0f03b69a2f085279b916f93be4d6cd42e75d3b038d8b9d9cee60c768efe3a1c2 Jul 23 15:41:14.793475 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.793386 2582 scope.go:117] "RemoveContainer" containerID="1843b22765ab9fdf647768ea4e61ddfb0aef3b7d5ddca431f8b95fe36664a46e" Jul 23 15:41:14.793475 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.793398 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:14.794866 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:14.794840 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425106448" event={"ID":"239314fd-f44b-4f6e-abe4-7e64eb3a3176","Type":"ContainerStarted","Data":"0f03b69a2f085279b916f93be4d6cd42e75d3b038d8b9d9cee60c768efe3a1c2"} Jul 23 15:41:15.803394 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:15.803356 2582 generic.go:356] "Generic (PLEG): container finished" podID="239314fd-f44b-4f6e-abe4-7e64eb3a3176" containerID="5e0f2f3f38e5d574a1954aac7f3af67903cfae9a85eaf8fca5c2f398f4f23bd4" exitCode=0 Jul 23 15:41:15.803770 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:15.803450 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7950-140710425106448" event={"ID":"239314fd-f44b-4f6e-abe4-7e64eb3a3176","Type":"ContainerDied","Data":"5e0f2f3f38e5d574a1954aac7f3af67903cfae9a85eaf8fca5c2f398f4f23bd4"} Jul 23 15:41:16.956486 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:16.956459 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:16.976981 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:16.976953 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7950-140710425106448_239314fd-f44b-4f6e-abe4-7e64eb3a3176/test-curl-7950-140710425106448/0.log" Jul 23 15:41:17.001650 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.001616 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:17.004081 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.004061 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7950-140710425106448"] Jul 23 15:41:17.057958 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.057928 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9trt\" (UniqueName: \"kubernetes.io/projected/239314fd-f44b-4f6e-abe4-7e64eb3a3176-kube-api-access-v9trt\") pod \"239314fd-f44b-4f6e-abe4-7e64eb3a3176\" (UID: \"239314fd-f44b-4f6e-abe4-7e64eb3a3176\") " Jul 23 15:41:17.060175 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.060122 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239314fd-f44b-4f6e-abe4-7e64eb3a3176-kube-api-access-v9trt" (OuterVolumeSpecName: "kube-api-access-v9trt") pod "239314fd-f44b-4f6e-abe4-7e64eb3a3176" (UID: "239314fd-f44b-4f6e-abe4-7e64eb3a3176"). InnerVolumeSpecName "kube-api-access-v9trt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:41:17.159195 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.159160 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9trt\" (UniqueName: \"kubernetes.io/projected/239314fd-f44b-4f6e-abe4-7e64eb3a3176-kube-api-access-v9trt\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:17.208579 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.208542 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58"] Jul 23 15:41:17.209012 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.208996 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239314fd-f44b-4f6e-abe4-7e64eb3a3176" containerName="test-curl-7950-140710425106448" Jul 23 15:41:17.209071 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.209015 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="239314fd-f44b-4f6e-abe4-7e64eb3a3176" containerName="test-curl-7950-140710425106448" Jul 23 15:41:17.209114 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.209104 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="239314fd-f44b-4f6e-abe4-7e64eb3a3176" containerName="test-curl-7950-140710425106448" Jul 23 15:41:17.212222 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.212206 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:17.215305 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.215251 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:41:17.218946 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.218920 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58"] Jul 23 15:41:17.361300 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.361194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58\" (UID: \"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:17.462370 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.462331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58\" (UID: \"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:17.462962 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.462940 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58\" (UID: \"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:17.524192 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.524150 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:17.662926 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.662898 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58"] Jul 23 15:41:17.663783 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:17.663757 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2cf71b_805c_4ce1_ad91_f8d2c2e70eec.slice/crio-0a0ad5f5bf62daa17b109257328345fde74bc8309f5867e9134681115dafbc7f WatchSource:0}: Error finding container 0a0ad5f5bf62daa17b109257328345fde74bc8309f5867e9134681115dafbc7f: Status 404 returned error can't find the container with id 0a0ad5f5bf62daa17b109257328345fde74bc8309f5867e9134681115dafbc7f Jul 23 15:41:17.817351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.817323 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:17.817351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.817362 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f03b69a2f085279b916f93be4d6cd42e75d3b038d8b9d9cee60c768efe3a1c2" Jul 23 15:41:17.818674 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:17.818641 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" event={"ID":"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec","Type":"ContainerStarted","Data":"0a0ad5f5bf62daa17b109257328345fde74bc8309f5867e9134681115dafbc7f"} Jul 23 15:41:17.827671 ip-10-0-142-11 kubenswrapper[2582]: E0723 15:41:17.827647 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7950-140710425106448\" is forbidden: User \"system:node:ip-10-0-142-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-11.ec2.internal' and this object" podUID="239314fd-f44b-4f6e-abe4-7e64eb3a3176" pod="opendatahub/test-curl-7950-140710425106448" Jul 23 15:41:18.295528 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:18.295487 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="239314fd-f44b-4f6e-abe4-7e64eb3a3176" path="/var/lib/kubelet/pods/239314fd-f44b-4f6e-abe4-7e64eb3a3176/volumes" Jul 23 15:41:18.825961 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:18.825925 2582 generic.go:356] "Generic (PLEG): container finished" podID="6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec" containerID="a04643017b9bc864f5573f4001e24479189971647f4aed27014a23df0c841dda" exitCode=0 Jul 23 15:41:18.826166 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:18.826013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" event={"ID":"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec","Type":"ContainerDied","Data":"a04643017b9bc864f5573f4001e24479189971647f4aed27014a23df0c841dda"} Jul 23 15:41:19.977077 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:19.977050 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:20.082802 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:20.082709 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec-maas-api-service-ca\") pod \"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec\" (UID: \"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec\") " Jul 23 15:41:20.083113 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:20.083088 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec" (UID: "6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:41:20.183950 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:20.183907 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:20.839127 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:20.839094 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" Jul 23 15:41:20.839330 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:20.839095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58" event={"ID":"6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec","Type":"ContainerDied","Data":"0a0ad5f5bf62daa17b109257328345fde74bc8309f5867e9134681115dafbc7f"} Jul 23 15:41:20.839330 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:20.839231 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a0ad5f5bf62daa17b109257328345fde74bc8309f5867e9134681115dafbc7f" Jul 23 15:41:32.280664 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:32.280633 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58"] Jul 23 15:41:32.284583 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:32.284541 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3e0e7-nbc58"] Jul 23 15:41:32.295597 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:32.295568 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec" path="/var/lib/kubelet/pods/6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec/volumes" Jul 23 15:41:33.163490 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:33.163463 2582 scope.go:117] "RemoveContainer" containerID="10985fed19bd30f71cd5a59bb0f3783eabc18626d039557e835b7caebebe30ee" Jul 23 15:41:39.187066 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.187031 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf"] Jul 23 15:41:39.187577 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.187485 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec" containerName="revoke-keys" Jul 23 15:41:39.187577 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.187498 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec" containerName="revoke-keys" Jul 23 15:41:39.187577 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.187560 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c2cf71b-805c-4ce1-ad91-f8d2c2e70eec" containerName="revoke-keys" Jul 23 15:41:39.192448 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.192427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:39.195571 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.195126 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6xkch\"" Jul 23 15:41:39.197103 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.197027 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf"] Jul 23 15:41:39.253301 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.253250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5552858f-0003-47bf-924f-be9a691b48c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf\" (UID: \"5552858f-0003-47bf-924f-be9a691b48c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:39.354339 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.354307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5552858f-0003-47bf-924f-be9a691b48c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf\" (UID: \"5552858f-0003-47bf-924f-be9a691b48c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:39.354898 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.354880 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5552858f-0003-47bf-924f-be9a691b48c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf\" (UID: \"5552858f-0003-47bf-924f-be9a691b48c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:39.506154 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.506062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:39.654768 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.654741 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf"] Jul 23 15:41:39.656660 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:41:39.656629 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5552858f_0003_47bf_924f_be9a691b48c6.slice/crio-385526e11974b994eab4ff536714f838b9acb298fa31e7c59707d433ef9c92b0 WatchSource:0}: Error finding container 385526e11974b994eab4ff536714f838b9acb298fa31e7c59707d433ef9c92b0: Status 404 returned error can't find the container with id 385526e11974b994eab4ff536714f838b9acb298fa31e7c59707d433ef9c92b0 Jul 23 15:41:39.958741 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:39.958707 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" event={"ID":"5552858f-0003-47bf-924f-be9a691b48c6","Type":"ContainerStarted","Data":"385526e11974b994eab4ff536714f838b9acb298fa31e7c59707d433ef9c92b0"} Jul 23 15:41:40.969204 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:40.969160 2582 generic.go:356] "Generic (PLEG): container finished" podID="5552858f-0003-47bf-924f-be9a691b48c6" containerID="d4901ff31e8d5f36bfc5345254c61162ea6becff0a52ac20cbaf6ea2c1f42221" exitCode=0 Jul 23 15:41:40.969723 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:40.969246 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" event={"ID":"5552858f-0003-47bf-924f-be9a691b48c6","Type":"ContainerDied","Data":"d4901ff31e8d5f36bfc5345254c61162ea6becff0a52ac20cbaf6ea2c1f42221"} Jul 23 15:41:42.118003 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.117972 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:42.179224 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.179192 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5552858f-0003-47bf-924f-be9a691b48c6-maas-api-service-ca\") pod \"5552858f-0003-47bf-924f-be9a691b48c6\" (UID: \"5552858f-0003-47bf-924f-be9a691b48c6\") " Jul 23 15:41:42.179676 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.179648 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5552858f-0003-47bf-924f-be9a691b48c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5552858f-0003-47bf-924f-be9a691b48c6" (UID: "5552858f-0003-47bf-924f-be9a691b48c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:41:42.280110 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.280008 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5552858f-0003-47bf-924f-be9a691b48c6-maas-api-service-ca\") on node \"ip-10-0-142-11.ec2.internal\" DevicePath \"\"" Jul 23 15:41:42.982667 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.982635 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" Jul 23 15:41:42.982667 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.982651 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf" event={"ID":"5552858f-0003-47bf-924f-be9a691b48c6","Type":"ContainerDied","Data":"385526e11974b994eab4ff536714f838b9acb298fa31e7c59707d433ef9c92b0"} Jul 23 15:41:42.982926 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:42.982699 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="385526e11974b994eab4ff536714f838b9acb298fa31e7c59707d433ef9c92b0" Jul 23 15:41:54.250020 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:54.249936 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf"] Jul 23 15:41:54.254780 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:54.254750 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-53a4f4-p92bf"] Jul 23 15:41:54.295142 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:41:54.295105 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5552858f-0003-47bf-924f-be9a691b48c6" path="/var/lib/kubelet/pods/5552858f-0003-47bf-924f-be9a691b48c6/volumes" Jul 23 15:42:07.551659 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:07.551626 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vbvdc_97bad5ce-6a36-484e-b9db-d017e9b81a8a/manager/0.log" Jul 23 15:42:07.684878 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:07.684839 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vb9b2_0d8257fa-d66f-4023-935a-f65d89354b55/manager/0.log" Jul 23 15:42:07.837120 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:07.837042 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-765b7646dc-qvtwr_ddec958d-b408-4737-9759-12f5cfb6384a/manager/0.log" Jul 23 15:42:07.967393 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:07.967361 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c4p9v_3288c6c1-8f5a-4c86-98b2-96c54c49dda0/server/0.log" Jul 23 15:42:08.102209 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:08.102134 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vgvfp_4ce15863-c157-45e4-bc18-c2b2b92df82b/manager/0.log" Jul 23 15:42:08.507720 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:08.507693 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b97c94c-wwbwv_ef5575ac-6836-4dda-81dc-e010dfeb5b3e/copy-manifests/0.log" Jul 23 15:42:08.521489 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:08.521460 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b97c94c-wwbwv_ef5575ac-6836-4dda-81dc-e010dfeb5b3e/manager/0.log" Jul 23 15:42:08.919637 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:08.919609 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-cb45bff9b-7tgjv_4acbb1ef-7753-46f8-b1d0-0f0aae5d4ca5/maas-api/0.log" Jul 23 15:42:09.047374 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:09.047344 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746995-qvwjx_d1569c61-30c2-4b25-bb48-8c3cf64422ac/cleanup/0.log" Jul 23 15:42:09.165650 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:09.165613 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747010-jrtct_8cc7600f-7338-4205-b235-281658cf46b1/cleanup/0.log" Jul 23 15:42:09.290214 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:09.290133 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-c48jr_5559df12-cb1f-4743-9d86-ab3dfa908ab1/postgres/0.log" Jul 23 15:42:10.238785 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.238760 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_800330fc-eb5c-455a-a0da-b2784726bf08/util/0.log" Jul 23 15:42:10.245896 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.245874 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_800330fc-eb5c-455a-a0da-b2784726bf08/pull/0.log" Jul 23 15:42:10.252999 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.252980 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_800330fc-eb5c-455a-a0da-b2784726bf08/extract/0.log" Jul 23 15:42:10.384153 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.384122 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk_5b95b16b-0ff8-4133-a082-823ab45a9bb4/pull/0.log" Jul 23 15:42:10.390471 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.390434 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk_5b95b16b-0ff8-4133-a082-823ab45a9bb4/extract/0.log" Jul 23 15:42:10.396351 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.396333 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk_5b95b16b-0ff8-4133-a082-823ab45a9bb4/util/0.log" Jul 23 15:42:10.559169 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.559051 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8_efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a/extract/0.log" Jul 23 15:42:10.565930 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.565905 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8_efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a/util/0.log" Jul 23 15:42:10.572602 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.572586 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8_efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a/pull/0.log" Jul 23 15:42:10.736107 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.736077 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7_ec980b45-51fc-4865-9eee-f00fec4061aa/util/0.log" Jul 23 15:42:10.743932 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.743912 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7_ec980b45-51fc-4865-9eee-f00fec4061aa/pull/0.log" Jul 23 15:42:10.750893 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:10.750876 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7_ec980b45-51fc-4865-9eee-f00fec4061aa/extract/0.log" Jul 23 15:42:11.070231 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:11.070195 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9xvpz_cc2286fa-1743-40a2-93a8-fa00ad471e92/manager/0.log" Jul 23 15:42:11.481014 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:11.480986 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6rxsf_e9f59e05-5769-48ae-9789-2e71a4a72928/registry-server/0.log" Jul 23 15:42:11.628062 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:11.628033 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tfkc7_b6379ab9-1845-4243-8cf6-3cd612f6df84/manager/0.log" Jul 23 15:42:12.128727 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:12.128699 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-n94sf_f9dc316e-b910-495e-8c00-302b1bfe9700/manager/0.log" Jul 23 15:42:14.073189 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:14.073154 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d89f64f-zpd4n_5dcc70a2-4867-4e0d-99fb-0005bd689a71/kube-auth-proxy/0.log" Jul 23 15:42:14.406278 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:14.406213 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-c5n7x_0b3d89a9-99ab-4766-8a3b-35be00431657/payload-pre-processing/0.log" Jul 23 15:42:14.550213 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:14.550184 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-48vkr_e3dacc07-d2bf-4d9c-a10d-50ca81f4d603/payload-processing/0.log" Jul 23 15:42:14.685106 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:14.685026 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f8bf867b-ckpfm_46c81778-15ff-474c-a0f3-10a6458b470a/router/0.log" Jul 23 15:42:15.243531 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:15.243502 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-29bdt_64ce1d79-7bb3-4585-8795-d75b04d24909/main/0.log" Jul 23 15:42:15.523402 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:15.523318 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwwwj2_fd561a8a-4d3d-400b-b136-9db8355361af/main/0.log" Jul 23 15:42:23.853643 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.853556 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8lzsc/must-gather-mqb6n"] Jul 23 15:42:23.854014 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.853997 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5552858f-0003-47bf-924f-be9a691b48c6" containerName="revoke-keys" Jul 23 15:42:23.854014 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.854009 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5552858f-0003-47bf-924f-be9a691b48c6" containerName="revoke-keys" Jul 23 15:42:23.854085 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.854074 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5552858f-0003-47bf-924f-be9a691b48c6" containerName="revoke-keys" Jul 23 15:42:23.857323 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.857306 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:23.859994 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.859966 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8lzsc\"/\"kube-root-ca.crt\"" Jul 23 15:42:23.861395 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.861374 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8lzsc\"/\"default-dockercfg-nn8hv\"" Jul 23 15:42:23.861525 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.861398 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8lzsc\"/\"openshift-service-ca.crt\"" Jul 23 15:42:23.864777 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.864436 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8lzsc/must-gather-mqb6n"] Jul 23 15:42:23.879421 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.879382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4tcx\" (UniqueName: \"kubernetes.io/projected/c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a-kube-api-access-s4tcx\") pod \"must-gather-mqb6n\" (UID: \"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a\") " pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:23.879421 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.879422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a-must-gather-output\") pod \"must-gather-mqb6n\" (UID: \"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a\") " pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:23.980504 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.980468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4tcx\" (UniqueName: \"kubernetes.io/projected/c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a-kube-api-access-s4tcx\") pod \"must-gather-mqb6n\" (UID: \"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a\") " pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:23.980504 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.980506 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a-must-gather-output\") pod \"must-gather-mqb6n\" (UID: \"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a\") " pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:23.980808 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.980793 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a-must-gather-output\") pod \"must-gather-mqb6n\" (UID: \"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a\") " pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:23.989342 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:23.989316 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4tcx\" (UniqueName: \"kubernetes.io/projected/c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a-kube-api-access-s4tcx\") pod \"must-gather-mqb6n\" (UID: \"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a\") " pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:24.167395 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:24.167368 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8lzsc/must-gather-mqb6n" Jul 23 15:42:24.319631 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:24.319600 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8lzsc/must-gather-mqb6n"] Jul 23 15:42:24.320823 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:42:24.320792 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0fc4795_01c1_4bdd_8d12_6ebfabba6a9a.slice/crio-08490ef9564a0b076ac577802bcd30f53c5fbf29226caddecb67262fc5fadd06 WatchSource:0}: Error finding container 08490ef9564a0b076ac577802bcd30f53c5fbf29226caddecb67262fc5fadd06: Status 404 returned error can't find the container with id 08490ef9564a0b076ac577802bcd30f53c5fbf29226caddecb67262fc5fadd06 Jul 23 15:42:25.237027 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:25.236991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lzsc/must-gather-mqb6n" event={"ID":"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a","Type":"ContainerStarted","Data":"08490ef9564a0b076ac577802bcd30f53c5fbf29226caddecb67262fc5fadd06"} Jul 23 15:42:26.247862 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:26.247816 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lzsc/must-gather-mqb6n" event={"ID":"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a","Type":"ContainerStarted","Data":"d189df35175e92a882838674221e99cb901b1bb5cd2a2dc028c08161c27573d9"} Jul 23 15:42:26.248367 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:26.247881 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lzsc/must-gather-mqb6n" event={"ID":"c0fc4795-01c1-4bdd-8d12-6ebfabba6a9a","Type":"ContainerStarted","Data":"61773e6d4f41d9a97411df39b5cac6554aa06b805cc6565802e8b55e7472942a"} Jul 23 15:42:26.265898 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:26.265816 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8lzsc/must-gather-mqb6n" podStartSLOduration=2.081960699 podStartE2EDuration="3.265795189s" totalImagesPullingTime="1.18383449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:42:26.264767209 +0000 UTC m=+2994.416216580" watchObservedRunningTime="2026-07-23 15:42:26.265795189 +0000 UTC m=+2994.417244558" Jul 23 15:42:27.004499 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:27.004447 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f2fgw_b8ac723b-5a09-485f-b5e9-faf6eef37039/global-pull-secret-syncer/0.log" Jul 23 15:42:27.156223 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:27.156190 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nqjm5_3a942429-26a7-434f-ba0f-8d7ec0691b1c/konnectivity-agent/0.log" Jul 23 15:42:27.226886 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:27.226851 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-11.ec2.internal_75c2663eec2141df3d5c4f1c1932094d/haproxy/0.log" Jul 23 15:42:31.885031 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:31.885004 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_800330fc-eb5c-455a-a0da-b2784726bf08/extract/0.log" Jul 23 15:42:31.910317 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:31.910225 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_800330fc-eb5c-455a-a0da-b2784726bf08/util/0.log" Jul 23 15:42:31.931623 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:31.931473 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599sqw6_800330fc-eb5c-455a-a0da-b2784726bf08/pull/0.log" Jul 23 15:42:31.959354 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:31.959318 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk_5b95b16b-0ff8-4133-a082-823ab45a9bb4/extract/0.log" Jul 23 15:42:31.979790 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:31.979709 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk_5b95b16b-0ff8-4133-a082-823ab45a9bb4/util/0.log" Jul 23 15:42:32.001369 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.001293 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0z2ksk_5b95b16b-0ff8-4133-a082-823ab45a9bb4/pull/0.log" Jul 23 15:42:32.043301 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.043274 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8_efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a/extract/0.log" Jul 23 15:42:32.077544 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.077507 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8_efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a/util/0.log" Jul 23 15:42:32.103408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.103273 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cjxw8_efe2ddb3-18ac-4fc8-8ee3-2e2bb85f856a/pull/0.log" Jul 23 15:42:32.133223 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.132858 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7_ec980b45-51fc-4865-9eee-f00fec4061aa/extract/0.log" Jul 23 15:42:32.178888 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.178860 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7_ec980b45-51fc-4865-9eee-f00fec4061aa/util/0.log" Jul 23 15:42:32.212951 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.212918 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bwnb7_ec980b45-51fc-4865-9eee-f00fec4061aa/pull/0.log" Jul 23 15:42:32.605955 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.605892 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9xvpz_cc2286fa-1743-40a2-93a8-fa00ad471e92/manager/0.log" Jul 23 15:42:32.674281 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.654092 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:42:32.674281 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.666765 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:42:32.754050 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.750436 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6rxsf_e9f59e05-5769-48ae-9789-2e71a4a72928/registry-server/0.log" Jul 23 15:42:32.955043 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:32.954970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tfkc7_b6379ab9-1845-4243-8cf6-3cd612f6df84/manager/0.log" Jul 23 15:42:33.067777 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:33.067735 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-n94sf_f9dc316e-b910-495e-8c00-302b1bfe9700/manager/0.log" Jul 23 15:42:34.875009 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:34.874934 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-58sm5_f90fd981-0d3a-4d04-adda-23df0f99ea87/cluster-monitoring-operator/0.log" Jul 23 15:42:35.008653 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.008615 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7bcf78d555-7rvnd_7083e62d-5c57-4533-848c-92a945ee2230/metrics-server/0.log" Jul 23 15:42:35.144203 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.144107 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kkpr5_a64b52a1-64f5-4159-9927-d77fd6f06584/node-exporter/0.log" Jul 23 15:42:35.186178 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.186153 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kkpr5_a64b52a1-64f5-4159-9927-d77fd6f06584/kube-rbac-proxy/0.log" Jul 23 15:42:35.218760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.218730 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kkpr5_a64b52a1-64f5-4159-9927-d77fd6f06584/init-textfile/0.log" Jul 23 15:42:35.759130 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.759101 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-vpdv9_819172c8-4d69-4e96-a60a-2dffbecf486a/prometheus-operator/0.log" Jul 23 15:42:35.782387 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.782357 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-vpdv9_819172c8-4d69-4e96-a60a-2dffbecf486a/kube-rbac-proxy/0.log" Jul 23 15:42:35.854868 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.854838 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69dd4f8bc4-rwtpf_80cd24b9-428c-446d-88da-ee579369c459/telemeter-client/0.log" Jul 23 15:42:35.887220 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.887193 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69dd4f8bc4-rwtpf_80cd24b9-428c-446d-88da-ee579369c459/reload/0.log" Jul 23 15:42:35.930363 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:35.930331 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69dd4f8bc4-rwtpf_80cd24b9-428c-446d-88da-ee579369c459/kube-rbac-proxy/0.log" Jul 23 15:42:37.185718 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.185680 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp"] Jul 23 15:42:37.193958 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.193772 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.196554 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.196525 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp"] Jul 23 15:42:37.230159 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.230132 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-9lsg8_08c05031-e76c-49da-94f0-a6329db3d717/networking-console-plugin/0.log" Jul 23 15:42:37.342672 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.342625 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-proc\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.343937 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.343896 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-podres\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.344307 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.344224 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-sys\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.344472 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.344456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-lib-modules\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.344669 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.344647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2dn\" (UniqueName: \"kubernetes.io/projected/1174da03-2e9a-4635-9ec4-06d1b766f5ee-kube-api-access-jv2dn\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446240 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446140 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-proc\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446240 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-podres\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-sys\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446303 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-lib-modules\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446315 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-proc\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446342 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2dn\" (UniqueName: \"kubernetes.io/projected/1174da03-2e9a-4635-9ec4-06d1b766f5ee-kube-api-access-jv2dn\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446349 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-podres\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446349 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-sys\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.446496 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.446425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1174da03-2e9a-4635-9ec4-06d1b766f5ee-lib-modules\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.466320 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.466284 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2dn\" (UniqueName: \"kubernetes.io/projected/1174da03-2e9a-4635-9ec4-06d1b766f5ee-kube-api-access-jv2dn\") pod \"perf-node-gather-daemonset-bgdbp\" (UID: \"1174da03-2e9a-4635-9ec4-06d1b766f5ee\") " pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.515760 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.515726 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:37.725138 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.724756 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp"] Jul 23 15:42:37.729300 ip-10-0-142-11 kubenswrapper[2582]: W0723 15:42:37.727247 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1174da03_2e9a_4635_9ec4_06d1b766f5ee.slice/crio-f51ab5bb3df93e1c582a6b07edccc15bcb77e219276da0f39d318ec90e271b1e WatchSource:0}: Error finding container f51ab5bb3df93e1c582a6b07edccc15bcb77e219276da0f39d318ec90e271b1e: Status 404 returned error can't find the container with id f51ab5bb3df93e1c582a6b07edccc15bcb77e219276da0f39d318ec90e271b1e Jul 23 15:42:37.872912 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.872877 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/1.log" Jul 23 15:42:37.879617 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:37.879033 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ctddv_097ab5e1-b387-48d5-a9a5-c4a604bd98bc/console-operator/2.log" Jul 23 15:42:38.367325 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:38.367202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" event={"ID":"1174da03-2e9a-4635-9ec4-06d1b766f5ee","Type":"ContainerStarted","Data":"1564ae41965331902646e547df9c6298b16aa327661835b495d518f9775ef9f8"} Jul 23 15:42:38.367325 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:38.367301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" event={"ID":"1174da03-2e9a-4635-9ec4-06d1b766f5ee","Type":"ContainerStarted","Data":"f51ab5bb3df93e1c582a6b07edccc15bcb77e219276da0f39d318ec90e271b1e"} Jul 23 15:42:38.368153 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:38.368129 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:38.385534 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:38.385478 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" podStartSLOduration=1.385460892 podStartE2EDuration="1.385460892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:42:38.383692426 +0000 UTC m=+3006.535141807" watchObservedRunningTime="2026-07-23 15:42:38.385460892 +0000 UTC m=+3006.536910261" Jul 23 15:42:38.392996 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:38.392966 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-55f9596797-w5hdr_78673a50-369a-4b33-8675-21dab537126c/console/0.log" Jul 23 15:42:39.027815 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:39.027780 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-vts29_8adfbf4b-b4e4-43cb-b2d0-72c75caa6939/volume-data-source-validator/0.log" Jul 23 15:42:40.006446 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:40.006405 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld8p4_9b3ffade-f266-426b-be05-2e4ee4eb13db/dns/0.log" Jul 23 15:42:40.027814 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:40.027790 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld8p4_9b3ffade-f266-426b-be05-2e4ee4eb13db/kube-rbac-proxy/0.log" Jul 23 15:42:40.157973 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:40.157936 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mfddd_d34f5e56-e8da-4a6a-9356-84a4ef68174a/dns-node-resolver/0.log" Jul 23 15:42:40.726611 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:40.726582 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8pjfm_bb31c270-e90d-4a01-99e1-5c3d39e63631/node-ca/0.log" Jul 23 15:42:41.837089 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:41.837052 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d89f64f-zpd4n_5dcc70a2-4867-4e0d-99fb-0005bd689a71/kube-auth-proxy/0.log" Jul 23 15:42:41.922615 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:41.922583 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-c5n7x_0b3d89a9-99ab-4766-8a3b-35be00431657/payload-pre-processing/0.log" Jul 23 15:42:41.954206 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:41.954176 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-48vkr_e3dacc07-d2bf-4d9c-a10d-50ca81f4d603/payload-processing/0.log" Jul 23 15:42:41.976840 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:41.976817 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f8bf867b-ckpfm_46c81778-15ff-474c-a0f3-10a6458b470a/router/0.log" Jul 23 15:42:42.664720 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:42.664689 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-67wsc_c15231de-2b65-4e56-8295-96311beed997/serve-healthcheck-canary/0.log" Jul 23 15:42:43.320590 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:43.320563 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-96lf4_b959cf26-685b-48c2-9b92-8969331ab09a/kube-rbac-proxy/0.log" Jul 23 15:42:43.340746 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:43.340719 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-96lf4_b959cf26-685b-48c2-9b92-8969331ab09a/exporter/0.log" Jul 23 15:42:43.361728 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:43.361702 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-96lf4_b959cf26-685b-48c2-9b92-8969331ab09a/extractor/0.log" Jul 23 15:42:45.394838 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:45.394805 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8lzsc/perf-node-gather-daemonset-bgdbp" Jul 23 15:42:45.457372 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:45.457343 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vbvdc_97bad5ce-6a36-484e-b9db-d017e9b81a8a/manager/0.log" Jul 23 15:42:45.709436 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:45.709402 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vb9b2_0d8257fa-d66f-4023-935a-f65d89354b55/manager/0.log" Jul 23 15:42:45.873128 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:45.873086 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-765b7646dc-qvtwr_ddec958d-b408-4737-9759-12f5cfb6384a/manager/0.log" Jul 23 15:42:45.896502 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:45.896475 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c4p9v_3288c6c1-8f5a-4c86-98b2-96c54c49dda0/server/0.log" Jul 23 15:42:45.980533 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:45.980459 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vgvfp_4ce15863-c157-45e4-bc18-c2b2b92df82b/manager/0.log" Jul 23 15:42:46.163855 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:46.163821 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b97c94c-wwbwv_ef5575ac-6836-4dda-81dc-e010dfeb5b3e/manager/0.log" Jul 23 15:42:46.181408 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:46.181369 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b97c94c-wwbwv_ef5575ac-6836-4dda-81dc-e010dfeb5b3e/copy-manifests/0.log" Jul 23 15:42:47.977869 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:47.977836 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d89f64f-zpd4n_5dcc70a2-4867-4e0d-99fb-0005bd689a71/kube-auth-proxy/0.log" Jul 23 15:42:48.077666 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:48.077620 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-c5n7x_0b3d89a9-99ab-4766-8a3b-35be00431657/payload-pre-processing/0.log" Jul 23 15:42:48.099790 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:48.099744 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-48vkr_e3dacc07-d2bf-4d9c-a10d-50ca81f4d603/payload-processing/0.log" Jul 23 15:42:48.114109 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:48.114077 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f8bf867b-ckpfm_46c81778-15ff-474c-a0f3-10a6458b470a/router/0.log" Jul 23 15:42:49.440273 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:49.440227 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8d89f64f-zpd4n_5dcc70a2-4867-4e0d-99fb-0005bd689a71/kube-auth-proxy/0.log" Jul 23 15:42:49.515055 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:49.515010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-c5n7x_0b3d89a9-99ab-4766-8a3b-35be00431657/payload-pre-processing/0.log" Jul 23 15:42:49.540943 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:49.540914 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-48vkr_e3dacc07-d2bf-4d9c-a10d-50ca81f4d603/payload-processing/0.log" Jul 23 15:42:49.556913 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:49.556873 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f8bf867b-ckpfm_46c81778-15ff-474c-a0f3-10a6458b470a/router/0.log" Jul 23 15:42:52.483900 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:52.483873 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-zk47q_5f04272a-404d-4370-96bb-019a750f3b49/migrator/0.log" Jul 23 15:42:52.503086 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:52.503053 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-zk47q_5f04272a-404d-4370-96bb-019a750f3b49/graceful-termination/0.log" Jul 23 15:42:53.941037 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:53.941005 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/kube-multus-additional-cni-plugins/0.log" Jul 23 15:42:53.961010 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:53.960979 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/egress-router-binary-copy/0.log" Jul 23 15:42:53.979937 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:53.979910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/cni-plugins/0.log" Jul 23 15:42:54.002025 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.001998 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/bond-cni-plugin/0.log" Jul 23 15:42:54.023768 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.023740 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/routeoverride-cni/0.log" Jul 23 15:42:54.046039 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.046003 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/whereabouts-cni-bincopy/0.log" Jul 23 15:42:54.066346 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.066322 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqkgc_f5a883e7-68b9-4878-af6d-68a66b399ce3/whereabouts-cni/0.log" Jul 23 15:42:54.257214 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.257138 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rfwvt_74d8f36b-9973-4647-a699-1c3ccf7fd19c/kube-multus/0.log" Jul 23 15:42:54.322929 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.322904 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xjwws_3d60ecea-1743-434c-8b1c-78b027daac9b/network-metrics-daemon/0.log" Jul 23 15:42:54.340187 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:54.340151 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xjwws_3d60ecea-1743-434c-8b1c-78b027daac9b/kube-rbac-proxy/0.log" Jul 23 15:42:55.168646 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.168618 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/ovn-controller/0.log" Jul 23 15:42:55.187867 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.187843 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/ovn-acl-logging/0.log" Jul 23 15:42:55.215793 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.215772 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/kube-rbac-proxy-node/0.log" Jul 23 15:42:55.237959 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.237927 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/kube-rbac-proxy-ovn-metrics/0.log" Jul 23 15:42:55.255276 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.255238 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/northd/0.log" Jul 23 15:42:55.277825 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.277799 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/nbdb/0.log" Jul 23 15:42:55.299529 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.299501 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/sbdb/0.log" Jul 23 15:42:55.364115 ip-10-0-142-11 kubenswrapper[2582]: I0723 15:42:55.364081 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2svvx_0181f318-c751-4ab6-bac0-cae5f18f87f0/ovnkube-controller/0.log"